[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:02 dist-test-slave-9tgz krb5kdc[13961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:02 dist-test-slave-9tgz krb5kdc[13961](info): set up 2 sockets
Sep 25 05:15:02 dist-test-slave-9tgz krb5kdc[13961](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 05:15:03.632961 13954 mini_kdc.cc:121] Time spent starting KDC: real 2.701s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260925 05:15:03.866001 13954 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.233s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260925 05:15:04.031808 13954 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.166s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260925 05:15:04.247870 13954 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.216s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/test-user.keytab.
W20260925 05:15:04.550046 13954 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.302s user 0.001s sys 0.001s
Sep 25 05:15:04 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:04.598270 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33317
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:04.684132 13974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:04.684420 13974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:04.684464 13974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:04.685743 13974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:04.685832 13974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:04.685855 13974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:04.685877 13974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:04.685909 13974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:04.687621 13974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:04.687947 13974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:04.688225 13974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:04.694006 13980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:04.694031 13982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:04.694059 13974 server_base.cc:1034] running on GCE node
W20260925 05:15:04.694042 13979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:04.694584 13974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:04.694939 13974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:04.696103 13974 hybrid_clock.cc:648] HybridClock initialized: now 1790313304696081 us; error 40 us; skew 500 ppm
Sep 25 05:15:04 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:04.698945 13974 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:04.699460 13974 webserver.cc:460] Webserver started at http://127.13.160.190:35091/ using document root <none> and password file <none>
I20260925 05:15:04.699692 13974 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:04.699738 13974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:04.699856 13974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:04.700996 13974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "dd2e4e089cc54a4ea316e7f63df6d08b"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "aa85f500eb80600a6d2cbd0e7a05f5f5"
server_key_iv: "4cf5e1c953fdd81e76c0e0efb35d6e1b"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.701321 13974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "dd2e4e089cc54a4ea316e7f63df6d08b"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "aa85f500eb80600a6d2cbd0e7a05f5f5"
server_key_iv: "4cf5e1c953fdd81e76c0e0efb35d6e1b"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.702728 13974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:04.703625 13989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:04.703832 13974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:04.703958 13974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "dd2e4e089cc54a4ea316e7f63df6d08b"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "aa85f500eb80600a6d2cbd0e7a05f5f5"
server_key_iv: "4cf5e1c953fdd81e76c0e0efb35d6e1b"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.704109 13974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:04.718726 13974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:04.721112 13974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:04.721308 13974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:04.725540 13974 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33317
I20260925 05:15:04.725584 14041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33317 every 8 connection(s)
I20260925 05:15:04.726008 13974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:04.727164 14042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:04.729923 14042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b: Bootstrap starting.
I20260925 05:15:04.730634 14042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:04.730993 14042 log.cc:824] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:04.731904 14042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b: No bootstrap required, opened a new log
I20260925 05:15:04.733726 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 13974
I20260925 05:15:04.733845 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:04.733935 14042 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } }
I20260925 05:15:04.734129 14042 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:04.734171 14042 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2e4e089cc54a4ea316e7f63df6d08b, State: Initialized, Role: FOLLOWER
I20260925 05:15:04.734330 14042 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } }
I20260925 05:15:04.734409 14042 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:04.734442 14042 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:04.734489 14042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:04.734658 13954 external_mini_cluster.cc:1428] Setting key 80afdf2ac1aa4a2047069724502fdfdf
I20260925 05:15:04.735450 14042 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } }
I20260925 05:15:04.735590 14042 leader_election.cc:302] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd2e4e089cc54a4ea316e7f63df6d08b; no voters:
I20260925 05:15:04.735811 14042 leader_election.cc:288] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:04.735898 14046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:04.736064 14046 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [term 1 LEADER]: Becoming Leader. State: Replica: dd2e4e089cc54a4ea316e7f63df6d08b, State: Running, Role: LEADER
I20260925 05:15:04.736178 14042 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:04.736223 14046 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } }
I20260925 05:15:04.736732 14048 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } } }
I20260925 05:15:04.736840 14048 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:04.736866 14049 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd2e4e089cc54a4ea316e7f63df6d08b. Latest consensus state: current_term: 1 leader_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2e4e089cc54a4ea316e7f63df6d08b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33317 } } }
I20260925 05:15:04.736987 14049 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:04.737344 14056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:04.737964 14056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:04.739869 14056 catalog_manager.cc:1347] Generated new cluster ID: 172f0cb044344149ac3e10f2e674b6d3
I20260925 05:15:04.739928 14056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:04 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:04.753048 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:04.735658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.13.160.190:33317)
0925 05:15:04.735909 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:04.735919 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:04.736349 (+ 430us) server_negotiation.cc:407] Connection header received
0925 05:15:04.739321 (+ 2972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:04.739329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:04.739389 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:04.739529 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:04.741294 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.742289 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.743513 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.744090 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.744263 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:04.748276 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:04.748289 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:04.748289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:04.748303 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:04.750275 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:04.750822 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:04.750827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:04.750827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:04.751006 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:04.751354 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:04.751355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:04.751355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:04.751758 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:04.751911 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:04.752484 (+ 573us) server_negotiation.cc:299] Negotiation successful
0925 05:15:04.752640 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":116,"threads_started":1}
I20260925 05:15:04.778271 14056 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:04.778878 14056 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:04.782213 14056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd2e4e089cc54a4ea316e7f63df6d08b: Generated new TSK 0
I20260925 05:15:04.782438 14056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:04.818324 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:04.921823 14068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:04.922075 14068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:04.922104 14068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:04.923301 14068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:04.923348 14068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:04.923379 14068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:04.925019 14068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:04.925725 14068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:04.925956 14068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:04.931319 14074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:04.931336 14073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:04.931316 14076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:04.931516 14068 server_base.cc:1034] running on GCE node
I20260925 05:15:04.931809 14068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:04.932034 14068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:04.933225 14068 hybrid_clock.cc:648] HybridClock initialized: now 1790313304933217 us; error 48 us; skew 500 ppm
Sep 25 05:15:04 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:04.935859 14068 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:04.936383 14068 webserver.cc:460] Webserver started at http://127.13.160.129:37193/ using document root <none> and password file <none>
I20260925 05:15:04.936620 14068 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:04.936691 14068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:04.936851 14068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:04.938066 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "78d627d716fe4b2e9c64597376c5808e"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "3d83bbcaefdd73a4bb769a53ae7d5d0e"
server_key_iv: "26bbac5592364a8cc2d92a4696c28ce8"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.938453 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "78d627d716fe4b2e9c64597376c5808e"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "3d83bbcaefdd73a4bb769a53ae7d5d0e"
server_key_iv: "26bbac5592364a8cc2d92a4696c28ce8"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.940032 14068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:04.940855 14083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:04.941095 14068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:04.941197 14068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "78d627d716fe4b2e9c64597376c5808e"
format_stamp: "Formatted at 2026-09-25 05:15:04 on dist-test-slave-9tgz"
server_key: "3d83bbcaefdd73a4bb769a53ae7d5d0e"
server_key_iv: "26bbac5592364a8cc2d92a4696c28ce8"
server_key_version: "encryptionkey@0"
I20260925 05:15:04.941277 14068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:04.966198 14068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:04.968327 14068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:04.968489 14068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:04.968757 14068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:04.969091 14068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:04.969126 14068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:04.969177 14068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:04.969208 14068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:04.975661 14068 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:42849
I20260925 05:15:04.975757 14196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:42849 every 8 connection(s)
I20260925 05:15:04.976111 14068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:04 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:04.984694 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14068
I20260925 05:15:04.984779 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:04.984966 13954 external_mini_cluster.cc:1428] Setting key 17a991e0c5f7598e915cb07984577724
I20260925 05:15:04.986122 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:04.976969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:49929 (local address 127.13.160.190:33317)
0925 05:15:04.977054 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:04.977056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:04.977266 (+ 210us) server_negotiation.cc:407] Connection header received
0925 05:15:04.977754 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:04.977755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:04.977766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:04.977829 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:04.978843 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.979426 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.980461 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.980674 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.980727 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:04.983168 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:04.983169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:04.983169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:04.983178 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:04.984413 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:04.984857 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:04.984858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:04.984859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:04.984924 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:04.985285 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:04.985286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:04.985287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:04.985442 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:04.985500 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:04.985867 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 05:15:04.985984 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 05:15:04.986163 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:04.976900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33317 (local address 127.13.160.129:49929)
0925 05:15:04.977124 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:04.977168 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:04.977665 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:04.977911 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:04.977915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:04.978534 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:04.978739 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:04.978746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.979522 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.979523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:04.980369 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:04.980371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:04.980752 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:04.980753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:04.980862 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:04.981485 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:04.981495 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:04.983077 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:04.984491 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:04.984499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:04.984502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:04.984735 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:04.985021 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:04.985022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:04.985023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:04.985183 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:04.985513 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:04.985521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:04.985703 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 05:15:04.985827 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 05:15:04.985975 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
I20260925 05:15:04.986757 14197 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33317
I20260925 05:15:04.986871 14197 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:04.987103 14197 heartbeater.cc:511] Master 127.13.160.190:33317 requested a full tablet report, sending...
I20260925 05:15:04.987804 14006 ts_manager.cc:194] Registered new tserver with Master: 78d627d716fe4b2e9c64597376c5808e (127.13.160.129:42849)
I20260925 05:15:04.988683 14006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:49929
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:05.010538 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:05.093891 14202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:05.094172 14202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:05.094215 14202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:05.095574 14202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:05.095640 14202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:05.095686 14202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:05.097283 14202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:05.097589 14202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:05.097851 14202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:05.103260 14210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:05.103410 14208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:05.103262 14207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:05.104178 14202 server_base.cc:1034] running on GCE node
I20260925 05:15:05.104414 14202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:05.104720 14202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:05.105908 14202 hybrid_clock.cc:648] HybridClock initialized: now 1790313305105880 us; error 85 us; skew 500 ppm
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:05.108726 14202 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:05.109242 14202 webserver.cc:460] Webserver started at http://127.13.160.130:43159/ using document root <none> and password file <none>
I20260925 05:15:05.109485 14202 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:05.109556 14202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:05.109707 14202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:05.110801 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "1e7a577fa7d143c6a923554dc0951d5e"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "86f3a756c0b5833497bac0daabdf1031"
server_key_iv: "87cc85fd769ba2eade2ce104fdccb447"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.111264 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "1e7a577fa7d143c6a923554dc0951d5e"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "86f3a756c0b5833497bac0daabdf1031"
server_key_iv: "87cc85fd769ba2eade2ce104fdccb447"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.112673 14202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:05.113435 14217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.113655 14202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:05.113751 14202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "1e7a577fa7d143c6a923554dc0951d5e"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "86f3a756c0b5833497bac0daabdf1031"
server_key_iv: "87cc85fd769ba2eade2ce104fdccb447"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.113806 14202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:05.131235 14202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:05.133291 14202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:05.133410 14202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:05.133602 14202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:05.133960 14202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:05.134001 14202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.134025 14202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:05.134073 14202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.140584 14202 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:42463
I20260925 05:15:05.140638 14330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:42463 every 8 connection(s)
I20260925 05:15:05.140992 14202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:05.146180 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14202
I20260925 05:15:05.146291 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:05.146461 13954 external_mini_cluster.cc:1428] Setting key acd98d7cea9fa91ebd90eaf081f53a1b
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:05.151706 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.141740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33317 (local address 127.13.160.130:39205)
0925 05:15:05.141889 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:05.141935 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:05.142431 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:05.142667 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:05.142671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:05.143344 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:05.143550 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:05.143556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.144266 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.144267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:05.145158 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:05.145160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.145651 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.145652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:05.145801 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.146463 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:05.146472 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:05.148184 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:05.150175 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:05.150181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:05.150183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:05.150435 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:05.150664 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:05.150664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:05.150664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:05.150764 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:05.151169 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:05.151174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:05.151306 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 05:15:05.151393 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 05:15:05.151524 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260925 05:15:05.151698 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.141755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39205 (local address 127.13.160.190:33317)
0925 05:15:05.141817 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:05.141819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:05.142031 (+ 212us) server_negotiation.cc:407] Connection header received
0925 05:15:05.142527 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:05.142528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:05.142539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:05.142597 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:05.143645 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.144168 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.145335 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.145560 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.145644 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.148300 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:05.148302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:05.148303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:05.148318 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:05.150083 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.150520 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.150522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.150522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.150593 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.150867 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.150869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.150869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.151085 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:05.151159 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:05.151442 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 05:15:05.151546 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 05:15:05.152282 14331 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33317
I20260925 05:15:05.152407 14331 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:05.152639 14331 heartbeater.cc:511] Master 127.13.160.190:33317 requested a full tablet report, sending...
I20260925 05:15:05.153081 14006 ts_manager.cc:194] Registered new tserver with Master: 1e7a577fa7d143c6a923554dc0951d5e (127.13.160.130:42463)
I20260925 05:15:05.153475 14006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:39205
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:05.173418 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:05.257174 14336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:05.257443 14336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:05.257489 14336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:05.258728 14336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:05.258808 14336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:05.258857 14336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:05.260492 14336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:05.260807 14336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:05.261058 14336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:05.266336 14342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:05.266319 14344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:05.266324 14341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:05.266597 14336 server_base.cc:1034] running on GCE node
I20260925 05:15:05.266821 14336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:05.267136 14336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:05.268281 14336 hybrid_clock.cc:648] HybridClock initialized: now 1790313305268271 us; error 28 us; skew 500 ppm
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:05.270853 14336 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:05.271433 14336 webserver.cc:460] Webserver started at http://127.13.160.131:43189/ using document root <none> and password file <none>
I20260925 05:15:05.271739 14336 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:05.271808 14336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:05.271950 14336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:05.273082 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "48bae90200da49c29662099e6ec7b070"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "3bf61a4b11e8e36ef278d823ade67a6d"
server_key_iv: "6a3c25013527f8f113432a91751a4ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.273471 14336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "48bae90200da49c29662099e6ec7b070"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "3bf61a4b11e8e36ef278d823ade67a6d"
server_key_iv: "6a3c25013527f8f113432a91751a4ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.274940 14336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:05.275741 14351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.275913 14336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:05.276016 14336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "48bae90200da49c29662099e6ec7b070"
format_stamp: "Formatted at 2026-09-25 05:15:05 on dist-test-slave-9tgz"
server_key: "3bf61a4b11e8e36ef278d823ade67a6d"
server_key_iv: "6a3c25013527f8f113432a91751a4ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:05.276104 14336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:05.291194 14336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:05.293202 14336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:05.293390 14336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:05.293655 14336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:05.293990 14336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:05.294047 14336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.294092 14336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:05.294116 14336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:05.300333 14336 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:35975
I20260925 05:15:05.300376 14464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:35975 every 8 connection(s)
I20260925 05:15:05.300767 14336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:05.308454 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14336
I20260925 05:15:05.308574 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:05.308768 13954 external_mini_cluster.cc:1428] Setting key 11dc30613bc2c944d852f20987cc5047
I20260925 05:15:05.310772 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.301571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36893 (local address 127.13.160.190:33317)
0925 05:15:05.301656 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:05.301660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:05.301891 (+ 231us) server_negotiation.cc:407] Connection header received
0925 05:15:05.302391 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:05.302392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:05.302403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:05.302459 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:05.303530 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.304027 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.305017 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.305216 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.305269 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.307533 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:05.307535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:05.307535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:05.307543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:05.309233 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.309737 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.309739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.309739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.309805 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.310078 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.310079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.310079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.310208 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:05.310286 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:05.310587 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 05:15:05.310676 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:15:05.311007 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.301547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33317 (local address 127.13.160.131:36893)
0925 05:15:05.301761 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:05.301801 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:05.302300 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:05.302522 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:05.302525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:05.303228 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:05.303438 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:05.303444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.304127 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.304128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:05.304940 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:05.304941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.305291 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.305292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:05.305399 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.305892 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:05.305901 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:05.307424 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:05.309339 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:05.309347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:05.309349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:05.309657 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:05.309874 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:05.309875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:05.309875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:05.310000 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:05.310290 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:05.310296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:05.310443 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 05:15:05.310551 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 05:15:05.310734 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":122,"threads_started":1}
I20260925 05:15:05.311488 14465 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33317
I20260925 05:15:05.311578 14465 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:05.311726 14465 heartbeater.cc:511] Master 127.13.160.190:33317 requested a full tablet report, sending...
I20260925 05:15:05.312045 14006 ts_manager.cc:194] Registered new tserver with Master: 48bae90200da49c29662099e6ec7b070 (127.13.160.131:35975)
I20260925 05:15:05.312441 14006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:36893
I20260925 05:15:05.321022 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:05.345510 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.337271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.13.160.129:42849)
0925 05:15:05.337431 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:05.337436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:05.337456 (+ 20us) server_negotiation.cc:407] Connection header received
0925 05:15:05.337484 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:05.337486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:05.337509 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:05.337581 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:05.338219 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.339031 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.339738 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.340069 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.340123 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.342378 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:05.342388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:05.342388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:05.342397 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:05.344192 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.344581 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.344589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.344590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.344666 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.344876 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.344877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.344877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.345035 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:05.345217 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:05.345290 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 05:15:05.345379 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260925 05:15:05.345911 14111 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48154
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[13961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:05.369095 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:05.361857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.13.160.129:42849)
0925 05:15:05.361945 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:05.361947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:05.362018 (+ 71us) server_negotiation.cc:407] Connection header received
0925 05:15:05.362070 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:05.362071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:05.362085 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:05.362140 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:05.362696 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.363275 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.363979 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:05.364175 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:05.364225 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:05.366627 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:05.366628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:05.366628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:05.366637 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:05.368152 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.368460 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.368461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.368461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.368514 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:05.368686 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:05.368686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:05.368687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:05.368787 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:05.368878 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:05.368951 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 05:15:05.369017 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 05:15:05.369666 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14068
I20260925 05:15:05.376003 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14202
I20260925 05:15:05.382434 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14336
I20260925 05:15:05.387578 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 13974
2026-09-25T05:15:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4482 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[14479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[14479](info): set up 2 sockets
Sep 25 05:15:05 dist-test-slave-9tgz krb5kdc[14479](info): commencing operation
krb5kdc: starting...
W20260925 05:15:06.423210 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:06.541219 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45607
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:45607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:06.621793 14492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:06.621994 14492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:06.622015 14492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:06.623198 14492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:06.623238 14492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:06.623252 14492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:06.623265 14492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:06.623311 14492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:06.624764 14492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:45607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:06.625011 14492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:06.625269 14492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:06.630002 14498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:06.630097 14492 server_base.cc:1034] running on GCE node
W20260925 05:15:06.630012 14497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:06.630020 14500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:06.630430 14492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:06.630731 14492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:06.631892 14492 hybrid_clock.cc:648] HybridClock initialized: now 1790313306631878 us; error 57 us; skew 500 ppm
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:06.634662 14492 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:06.635156 14492 webserver.cc:460] Webserver started at http://127.13.160.190:44821/ using document root <none> and password file <none>
I20260925 05:15:06.635402 14492 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:06.635447 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:06.635624 14492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:06.636686 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "a26959c44b7d4603ad591cf0ad6d06db"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "45d1a4c6498dad534e613aeb62a05aee"
server_key_iv: "1118d72e5f9cde470674929aee0f7722"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.637050 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "a26959c44b7d4603ad591cf0ad6d06db"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "45d1a4c6498dad534e613aeb62a05aee"
server_key_iv: "1118d72e5f9cde470674929aee0f7722"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.638346 14492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:06.639081 14507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:06.639259 14492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:06.639358 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "a26959c44b7d4603ad591cf0ad6d06db"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "45d1a4c6498dad534e613aeb62a05aee"
server_key_iv: "1118d72e5f9cde470674929aee0f7722"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.639441 14492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:06.653766 14492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:06.656260 14492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:06.656430 14492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:06.660595 14492 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:45607
I20260925 05:15:06.660657 14559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:45607 every 8 connection(s)
I20260925 05:15:06.661023 14492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:06.661912 14560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:06.664378 14560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db: Bootstrap starting.
I20260925 05:15:06.665021 14560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:06.665355 14560 log.cc:824] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:06.666066 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14492
I20260925 05:15:06.666165 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:06.666177 14560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db: No bootstrap required, opened a new log
I20260925 05:15:06.666345 13954 external_mini_cluster.cc:1428] Setting key 6ffb8eec63a78779644b10c1488a70c4
I20260925 05:15:06.668819 14560 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } }
I20260925 05:15:06.668962 14560 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:06.668987 14560 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26959c44b7d4603ad591cf0ad6d06db, State: Initialized, Role: FOLLOWER
I20260925 05:15:06.669127 14560 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } }
I20260925 05:15:06.669210 14560 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:06.669234 14560 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:06.669308 14560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:06.669922 14560 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } }
I20260925 05:15:06.670043 14560 leader_election.cc:302] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26959c44b7d4603ad591cf0ad6d06db; no voters:
I20260925 05:15:06.670220 14560 leader_election.cc:288] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:06.670284 14565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:06.670437 14565 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [term 1 LEADER]: Becoming Leader. State: Replica: a26959c44b7d4603ad591cf0ad6d06db, State: Running, Role: LEADER
I20260925 05:15:06.670502 14560 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:06.670521 14565 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } }
I20260925 05:15:06.670858 14565 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26959c44b7d4603ad591cf0ad6d06db. Latest consensus state: current_term: 1 leader_uuid: "a26959c44b7d4603ad591cf0ad6d06db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } } }
I20260925 05:15:06.670997 14565 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:06.670869 14566 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26959c44b7d4603ad591cf0ad6d06db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26959c44b7d4603ad591cf0ad6d06db" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45607 } } }
I20260925 05:15:06.671108 14566 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:06.671202 14571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:06.671697 14571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:06.673221 14571 catalog_manager.cc:1347] Generated new cluster ID: 515ffd758921499cb011c8169cdd4f50
I20260925 05:15:06.673280 14571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:06.678984 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:06.666868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.13.160.190:45607)
0925 05:15:06.667047 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:06.667055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:06.667089 (+ 34us) server_negotiation.cc:407] Connection header received
0925 05:15:06.667662 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:06.667670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:06.667717 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:06.667829 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:06.668633 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.669950 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.671243 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.672017 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.672118 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:06.675186 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:06.675190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:06.675190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:06.675202 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:06.677250 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:06.677688 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:06.677691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:06.677691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:06.677798 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:06.678014 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:06.678015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:06.678015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:06.678340 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:06.678498 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:06.678619 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 05:15:06.678756 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260925 05:15:06.688683 14571 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:06.689224 14571 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:06.698114 14571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26959c44b7d4603ad591cf0ad6d06db: Generated new TSK 0
I20260925 05:15:06.698303 14571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:06.726141 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:06.806936 14586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:06.807147 14586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:06.807171 14586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:06.808271 14586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:06.808306 14586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:06.808331 14586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:06.809724 14586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:06.809916 14586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:06.810093 14586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:06.814998 14592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:06.815022 14591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:06.815060 14594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:06.815377 14586 server_base.cc:1034] running on GCE node
I20260925 05:15:06.815606 14586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:06.815862 14586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:06.817058 14586 hybrid_clock.cc:648] HybridClock initialized: now 1790313306817041 us; error 29 us; skew 500 ppm
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:06.819629 14586 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:06.820060 14586 webserver.cc:460] Webserver started at http://127.13.160.129:34753/ using document root <none> and password file <none>
I20260925 05:15:06.820297 14586 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:06.820364 14586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:06.820537 14586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:06.821569 14586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "c56cbebc4ad043e9ad5155e598d6763d"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "532331cb70d314a5ab7caba547b171a5"
server_key_iv: "bec58615613d814ec77f3cfe45e06761"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.821926 14586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "c56cbebc4ad043e9ad5155e598d6763d"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "532331cb70d314a5ab7caba547b171a5"
server_key_iv: "bec58615613d814ec77f3cfe45e06761"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.823319 14586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:06.824038 14601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:06.824213 14586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:06.824311 14586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "c56cbebc4ad043e9ad5155e598d6763d"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "532331cb70d314a5ab7caba547b171a5"
server_key_iv: "bec58615613d814ec77f3cfe45e06761"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.824396 14586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:06.842078 14586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:06.844249 14586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:06.844377 14586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:06.844590 14586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:06.844842 14586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:06.844884 14586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:06.844905 14586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:06.844918 14586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:06.850493 14586 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40065
I20260925 05:15:06.850561 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40065 every 8 connection(s)
I20260925 05:15:06.850979 14586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:06.860580 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:06.851735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:44223 (local address 127.13.160.190:45607)
0925 05:15:06.851796 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:06.851798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:06.852021 (+ 223us) server_negotiation.cc:407] Connection header received
0925 05:15:06.852929 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:06.852930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:06.852940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:06.852999 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:06.853579 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.854077 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.855629 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.855827 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.855886 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:06.858075 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:06.858076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:06.858076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:06.858082 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:06.859385 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:06.859763 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:06.859764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:06.859764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:06.859823 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:06.860040 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:06.860041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:06.860041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:06.860148 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:06.860201 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:06.860421 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 05:15:06.860511 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 05:15:06.860710 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:06.851741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45607 (local address 127.13.160.129:44223)
0925 05:15:06.851898 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:06.851933 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:06.852840 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:06.853057 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:06.853060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:06.853315 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:06.853509 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:06.853513 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.854152 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.854153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:06.855541 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:06.855544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:06.855901 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:06.855902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:06.856001 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:06.856564 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:06.856579 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:06.857992 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:06.859466 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:06.859472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:06.859474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:06.859681 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:06.859891 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:06.859892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:06.859892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:06.859985 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:06.860203 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:06.860207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:06.860318 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 05:15:06.860401 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 05:15:06.860517 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260925 05:15:06.860930 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14586
I20260925 05:15:06.861014 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:06.861192 13954 external_mini_cluster.cc:1428] Setting key 79091be15af93e8f8156818f6d9b5b8f
I20260925 05:15:06.861246 14715 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45607
I20260925 05:15:06.861337 14715 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:06.861542 14715 heartbeater.cc:511] Master 127.13.160.190:45607 requested a full tablet report, sending...
I20260925 05:15:06.862059 14524 ts_manager.cc:194] Registered new tserver with Master: c56cbebc4ad043e9ad5155e598d6763d (127.13.160.129:40065)
I20260925 05:15:06.862664 14524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:44223
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:06.885804 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:06.966984 14720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:06.967196 14720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:06.967221 14720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:06.968389 14720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:06.968434 14720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:06.968497 14720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:06.970000 14720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:06.970245 14720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:06.970490 14720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:06.975744 14726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:06.975783 14725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:06.975756 14728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:06.975783 14720 server_base.cc:1034] running on GCE node
I20260925 05:15:06.976152 14720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:06.976410 14720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:06.977610 14720 hybrid_clock.cc:648] HybridClock initialized: now 1790313306977597 us; error 41 us; skew 500 ppm
Sep 25 05:15:06 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:06.986081 14720 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:06.986654 14720 webserver.cc:460] Webserver started at http://127.13.160.130:43839/ using document root <none> and password file <none>
I20260925 05:15:06.986972 14720 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:06.987047 14720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:06.987213 14720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:06.988355 14720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "8de82d73aaaf441daa68c50ae53acec7"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "56f49aec6097644c6bcb8cb6b925efe6"
server_key_iv: "f5041b0e3608d396bad88e77184c7f9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.988763 14720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "8de82d73aaaf441daa68c50ae53acec7"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "56f49aec6097644c6bcb8cb6b925efe6"
server_key_iv: "f5041b0e3608d396bad88e77184c7f9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.990208 14720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:06.991057 14735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:06.991241 14720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:06.991348 14720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "8de82d73aaaf441daa68c50ae53acec7"
format_stamp: "Formatted at 2026-09-25 05:15:06 on dist-test-slave-9tgz"
server_key: "56f49aec6097644c6bcb8cb6b925efe6"
server_key_iv: "f5041b0e3608d396bad88e77184c7f9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:06.991433 14720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:07.023059 14720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:07.025590 14720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:07.025763 14720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:07.026012 14720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:07.026340 14720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:07.026396 14720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:07.026438 14720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:07.026494 14720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:07.032593 14720 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:33295
I20260925 05:15:07.032624 14848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:33295 every 8 connection(s)
I20260925 05:15:07.032963 14720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:07.040760 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14720
I20260925 05:15:07.040859 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:07.041038 13954 external_mini_cluster.cc:1428] Setting key 7cdeb0c64abd4e6641e1a69c930fc5cc
I20260925 05:15:07.042904 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.033834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:34795 (local address 127.13.160.190:45607)
0925 05:15:07.033900 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.033902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.034146 (+ 244us) server_negotiation.cc:407] Connection header received
0925 05:15:07.035122 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.035123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.035132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.035194 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.035830 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.036315 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.037265 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.037441 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.037491 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.039656 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.039663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.039663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.039669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.041288 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.041778 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.041779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.041780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.041861 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.042164 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.042165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.042166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.042318 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.042391 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.042630 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.042720 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:15:07.042979 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.033802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45607 (local address 127.13.160.130:34795)
0925 05:15:07.034002 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.034044 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.035031 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.035252 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.035255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.035509 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:07.035758 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.035762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.036389 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.036390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.037192 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.037194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.037509 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.037510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.037617 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.038093 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:07.038101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:07.039545 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:07.041376 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:07.041387 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:07.041389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:07.041704 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:07.041965 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:07.041966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:07.041966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:07.042095 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:07.042382 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:07.042385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:07.042499 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.042590 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.042713 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260925 05:15:07.043458 14849 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45607
I20260925 05:15:07.043571 14849 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:07.043771 14849 heartbeater.cc:511] Master 127.13.160.190:45607 requested a full tablet report, sending...
I20260925 05:15:07.044147 14524 ts_manager.cc:194] Registered new tserver with Master: 8de82d73aaaf441daa68c50ae53acec7 (127.13.160.130:33295)
I20260925 05:15:07.044513 14524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:34795
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:07.065995 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:07.149866 14854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:07.150110 14854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:07.150167 14854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:07.151419 14854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:07.151468 14854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:07.151497 14854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:07.152978 14854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45607
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:07.153204 14854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:07.153391 14854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:07.158555 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:07.158524 14859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:07.158574 14860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:07.158676 14854 server_base.cc:1034] running on GCE node
I20260925 05:15:07.158993 14854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:07.159269 14854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:07.160435 14854 hybrid_clock.cc:648] HybridClock initialized: now 1790313307160427 us; error 35 us; skew 500 ppm
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:07.163133 14854 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:07.163609 14854 webserver.cc:460] Webserver started at http://127.13.160.131:46691/ using document root <none> and password file <none>
I20260925 05:15:07.163859 14854 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:07.163931 14854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:07.164095 14854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:07.165179 14854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "149d5dc96c81470e94412e90c8cd7efc"
format_stamp: "Formatted at 2026-09-25 05:15:07 on dist-test-slave-9tgz"
server_key: "be06675dfd9b37eb51a5ba6e28cfecd9"
server_key_iv: "6344f63160d3b69e00c279b1d49ab4ff"
server_key_version: "encryptionkey@0"
I20260925 05:15:07.165567 14854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "149d5dc96c81470e94412e90c8cd7efc"
format_stamp: "Formatted at 2026-09-25 05:15:07 on dist-test-slave-9tgz"
server_key: "be06675dfd9b37eb51a5ba6e28cfecd9"
server_key_iv: "6344f63160d3b69e00c279b1d49ab4ff"
server_key_version: "encryptionkey@0"
I20260925 05:15:07.167044 14854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:15:07.167819 14869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:07.167987 14854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:07.168099 14854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "149d5dc96c81470e94412e90c8cd7efc"
format_stamp: "Formatted at 2026-09-25 05:15:07 on dist-test-slave-9tgz"
server_key: "be06675dfd9b37eb51a5ba6e28cfecd9"
server_key_iv: "6344f63160d3b69e00c279b1d49ab4ff"
server_key_version: "encryptionkey@0"
I20260925 05:15:07.168191 14854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:07.184377 14854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:07.186403 14854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:07.186579 14854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:07.186826 14854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:07.187178 14854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:07.187237 14854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:07.187278 14854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:07.187306 14854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:07.193177 14854 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:39837
I20260925 05:15:07.193260 14982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:39837 every 8 connection(s)
I20260925 05:15:07.193611 14854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:07.200977 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 14854
I20260925 05:15:07.201099 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:07.201290 13954 external_mini_cluster.cc:1428] Setting key 942c4d77d7b11dc17b8f904402e5c6f3
I20260925 05:15:07.203698 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.194443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:57113 (local address 127.13.160.190:45607)
0925 05:15:07.194524 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.194526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.194733 (+ 207us) server_negotiation.cc:407] Connection header received
0925 05:15:07.195832 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.195833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.195844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.195901 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.196509 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.197006 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.198051 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.198363 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.198429 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.200637 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.200639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.200639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.200647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.202450 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.202825 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.202826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.202826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.202878 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.203156 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.203157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.203157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.203267 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.203318 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.203568 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.203630 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:15:07.203889 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.194447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45607 (local address 127.13.160.131:57113)
0925 05:15:07.194601 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.194640 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.195727 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.195960 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.195964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.196233 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:07.196432 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.196437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.197095 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.197095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.197951 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.197954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.198440 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.198441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.198570 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.199126 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:07.199135 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:07.200546 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:07.202522 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:07.202527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:07.202529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:07.202760 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:07.202970 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:07.202971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:07.202971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:07.203100 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:07.203323 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:07.203327 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:07.203452 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.203542 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.203666 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260925 05:15:07.204367 14983 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45607
I20260925 05:15:07.204489 14983 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:07.204679 14983 heartbeater.cc:511] Master 127.13.160.190:45607 requested a full tablet report, sending...
I20260925 05:15:07.205044 14524 ts_manager.cc:194] Registered new tserver with Master: 149d5dc96c81470e94412e90c8cd7efc (127.13.160.131:39837)
I20260925 05:15:07.205432 14524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:57113
I20260925 05:15:07.213121 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:07.221021 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.214803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.13.160.190:45607)
0925 05:15:07.214889 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.214892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.215026 (+ 134us) server_negotiation.cc:407] Connection header received
0925 05:15:07.215124 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.215125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.215137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.215183 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.215773 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.216276 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.217011 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.217224 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.217297 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.218459 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.218461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.218461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.218469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.219705 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.220066 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.220067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.220068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.220129 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.220340 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.220342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.220342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.220542 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.220670 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.220810 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.220899 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 05:15:07.223313 14524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:07.225006 14524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:07.232227 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40065 (local address 127.0.0.1:49686)
0925 05:15:07.227961 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.227971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.228011 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.228276 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.228276 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.228278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:07.228317 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.228318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229365 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.229366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.230706 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.230708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.231878 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.231879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.231964 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.231980 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.232066 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.232131 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":32,"thread_start_us":88,"threads_started":1}
I20260925 05:15:07.232235 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.13.160.129:40065)
0925 05:15:07.228121 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.228126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.228136 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:07.228178 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.228181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.228196 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.228278 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:07.228480 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229250 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.230808 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.231803 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.231880 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.231941 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.232105 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.232150 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260925 05:15:07.233114 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:39837 (local address 127.0.0.1:56252)
0925 05:15:07.227670 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.227712 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.227777 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.228474 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.228475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.228477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:07.228510 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.228511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229729 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.229730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.232031 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.232033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.232851 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.232852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.232924 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.232928 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.232986 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.233028 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":110,"threads_started":1}
I20260925 05:15:07.233119 14996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.13.160.131:39837)
0925 05:15:07.228132 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.228136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.228146 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:07.228180 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.228182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.228196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.228277 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:07.228587 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229629 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.232122 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.232762 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.232853 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.232904 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.233002 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.233042 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":74,"threads_started":1}
I20260925 05:15:07.233115 14649 tablet_service.cc:1467] Processing CreateTablet for tablet 77cecc21735f4f85beccc9db1d3bc8f3 (DEFAULT_TABLE table=test-table [id=9eb9ad9dc5df4beebdf3a8ca9b614c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:07.233546 14649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cecc21735f4f85beccc9db1d3bc8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:07.233747 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33295 (local address 127.0.0.1:53562)
0925 05:15:07.227867 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.227878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.227936 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.228150 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.228154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.228157 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:07.228230 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.228232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229600 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.229601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.230746 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.230747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.233504 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.233505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.233577 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.233581 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.233637 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.233678 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":28,"thread_start_us":77,"threads_started":1}
I20260925 05:15:07.233816 14917 tablet_service.cc:1467] Processing CreateTablet for tablet 77cecc21735f4f85beccc9db1d3bc8f3 (DEFAULT_TABLE table=test-table [id=9eb9ad9dc5df4beebdf3a8ca9b614c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:07.234043 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.227708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.13.160.130:33295)
0925 05:15:07.227906 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.227912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.227923 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:07.228033 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.228037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.228055 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.228137 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:07.228314 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.229472 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.230821 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.233095 (+ 2274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.233488 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.233560 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.233919 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.233970 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260925 05:15:07.234175 14917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cecc21735f4f85beccc9db1d3bc8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:07.234771 14783 tablet_service.cc:1467] Processing CreateTablet for tablet 77cecc21735f4f85beccc9db1d3bc8f3 (DEFAULT_TABLE table=test-table [id=9eb9ad9dc5df4beebdf3a8ca9b614c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:07.235116 14783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77cecc21735f4f85beccc9db1d3bc8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:07.237206 15000 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Bootstrap starting.
I20260925 05:15:07.237638 15001 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: Bootstrap starting.
I20260925 05:15:07.238050 15000 tablet_bootstrap.cc:654] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:07.238269 15001 tablet_bootstrap.cc:654] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:07.238310 15002 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Bootstrap starting.
I20260925 05:15:07.238665 15000 log.cc:824] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:07.238852 15001 log.cc:824] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:07.239089 15002 tablet_bootstrap.cc:654] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:07.239624 15000 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: No bootstrap required, opened a new log
I20260925 05:15:07.239713 15000 ts_tablet_manager.cc:1397] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260925 05:15:07.239729 15001 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: No bootstrap required, opened a new log
I20260925 05:15:07.239717 15002 log.cc:824] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:07.239784 15001 ts_tablet_manager.cc:1397] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:07.240495 15002 tablet_bootstrap.cc:492] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: No bootstrap required, opened a new log
I20260925 05:15:07.240569 15002 ts_tablet_manager.cc:1397] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:07.241456 15000 raft_consensus.cc:348] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.241670 15000 raft_consensus.cc:374] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:07.241715 15000 raft_consensus.cc:729] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56cbebc4ad043e9ad5155e598d6763d, State: Initialized, Role: FOLLOWER
I20260925 05:15:07.241828 15000 consensus_queue.cc:260] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.241780 15001 raft_consensus.cc:348] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.241942 15001 raft_consensus.cc:374] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:07.241983 15001 raft_consensus.cc:729] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149d5dc96c81470e94412e90c8cd7efc, State: Initialized, Role: FOLLOWER
I20260925 05:15:07.242143 15000 ts_tablet_manager.cc:1428] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 05:15:07.242117 15001 consensus_queue.cc:260] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.242179 14715 heartbeater.cc:503] Master 127.13.160.190:45607 was elected leader, sending a full tablet report...
I20260925 05:15:07.242368 15001 ts_tablet_manager.cc:1428] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:07.242487 14983 heartbeater.cc:503] Master 127.13.160.190:45607 was elected leader, sending a full tablet report...
I20260925 05:15:07.242727 15002 raft_consensus.cc:348] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.242976 15002 raft_consensus.cc:374] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:07.243022 15002 raft_consensus.cc:729] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de82d73aaaf441daa68c50ae53acec7, State: Initialized, Role: FOLLOWER
I20260925 05:15:07.243125 15002 consensus_queue.cc:260] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.243399 14849 heartbeater.cc:503] Master 127.13.160.190:45607 was elected leader, sending a full tablet report...
I20260925 05:15:07.243394 15002 ts_tablet_manager.cc:1428] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260925 05:15:07.283540 14850 tablet.cc:2369] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:07.311609 15007 raft_consensus.cc:482] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:07.311738 15007 raft_consensus.cc:504] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.312157 15007 leader_election.cc:288] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8de82d73aaaf441daa68c50ae53acec7 (127.13.160.130:33295), c56cbebc4ad043e9ad5155e598d6763d (127.13.160.129:40065)
I20260925 05:15:07.315418 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.312427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:33959 (local address 127.13.160.129:40065)
0925 05:15:07.312478 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.312481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.312491 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:07.312543 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.312544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.312555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.312631 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:07.312883 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.313459 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.314437 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.315048 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.315123 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.315132 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.315284 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.315345 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:15:07.315395 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.312343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40065 (local address 127.13.160.131:33959)
0925 05:15:07.312402 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.312414 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.312455 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.312708 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.312709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.312711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:07.312797 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.312799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.313543 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.313544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.314308 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.314310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.315119 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.315120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.315179 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.315182 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.315257 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.315303 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 05:15:07.315480 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.312491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:38429 (local address 127.13.160.130:33295)
0925 05:15:07.312541 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.312542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.312688 (+ 146us) server_negotiation.cc:407] Connection header received
0925 05:15:07.312715 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.312716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.312724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.312786 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:07.312957 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.313458 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.314564 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.315155 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.315224 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.315230 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.315371 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.315410 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 05:15:07.315788 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.312384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33295 (local address 127.13.160.131:38429)
0925 05:15:07.312596 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:07.312606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:07.312662 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:07.312797 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:07.312798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:07.312799 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:07.312884 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.312886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.313571 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.313572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.314449 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:07.314452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.315226 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.315226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:07.315294 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.315297 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:07.315350 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 05:15:07.315380 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":130,"threads_started":1}
I20260925 05:15:07.315941 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cecc21735f4f85beccc9db1d3bc8f3" candidate_uuid: "149d5dc96c81470e94412e90c8cd7efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56cbebc4ad043e9ad5155e598d6763d" is_pre_election: true
I20260925 05:15:07.316144 14669 raft_consensus.cc:2454] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149d5dc96c81470e94412e90c8cd7efc in term 0.
I20260925 05:15:07.316155 14803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cecc21735f4f85beccc9db1d3bc8f3" candidate_uuid: "149d5dc96c81470e94412e90c8cd7efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de82d73aaaf441daa68c50ae53acec7" is_pre_election: true
I20260925 05:15:07.316308 14803 raft_consensus.cc:2454] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149d5dc96c81470e94412e90c8cd7efc in term 0.
I20260925 05:15:07.316414 14870 leader_election.cc:302] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149d5dc96c81470e94412e90c8cd7efc, c56cbebc4ad043e9ad5155e598d6763d; no voters:
I20260925 05:15:07.316558 15007 raft_consensus.cc:2781] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:07.316615 15007 raft_consensus.cc:482] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:07.316658 15007 raft_consensus.cc:3037] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:07.317381 15007 raft_consensus.cc:504] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.317535 15007 leader_election.cc:288] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [CANDIDATE]: Term 1 election: Requested vote from peers 8de82d73aaaf441daa68c50ae53acec7 (127.13.160.130:33295), c56cbebc4ad043e9ad5155e598d6763d (127.13.160.129:40065)
I20260925 05:15:07.317724 14669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cecc21735f4f85beccc9db1d3bc8f3" candidate_uuid: "149d5dc96c81470e94412e90c8cd7efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56cbebc4ad043e9ad5155e598d6763d"
I20260925 05:15:07.317732 14803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77cecc21735f4f85beccc9db1d3bc8f3" candidate_uuid: "149d5dc96c81470e94412e90c8cd7efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de82d73aaaf441daa68c50ae53acec7"
I20260925 05:15:07.317809 14669 raft_consensus.cc:3037] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:07.317827 14803 raft_consensus.cc:3037] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:07.318555 14803 raft_consensus.cc:2454] T 77cecc21735f4f85beccc9db1d3bc8f3 P 8de82d73aaaf441daa68c50ae53acec7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149d5dc96c81470e94412e90c8cd7efc in term 1.
I20260925 05:15:07.318606 14669 raft_consensus.cc:2454] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149d5dc96c81470e94412e90c8cd7efc in term 1.
I20260925 05:15:07.318770 14872 leader_election.cc:302] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149d5dc96c81470e94412e90c8cd7efc, 8de82d73aaaf441daa68c50ae53acec7; no voters:
I20260925 05:15:07.318871 15007 raft_consensus.cc:2781] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:07.319125 15007 raft_consensus.cc:686] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [term 1 LEADER]: Becoming Leader. State: Replica: 149d5dc96c81470e94412e90c8cd7efc, State: Running, Role: LEADER
I20260925 05:15:07.319229 15007 consensus_queue.cc:237] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } }
I20260925 05:15:07.320151 14523 catalog_manager.cc:5495] T 77cecc21735f4f85beccc9db1d3bc8f3 P 149d5dc96c81470e94412e90c8cd7efc reported cstate change: term changed from 0 to 1, leader changed from <none> to 149d5dc96c81470e94412e90c8cd7efc (127.13.160.131). New cstate: current_term: 1 leader_uuid: "149d5dc96c81470e94412e90c8cd7efc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8de82d73aaaf441daa68c50ae53acec7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149d5dc96c81470e94412e90c8cd7efc" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c56cbebc4ad043e9ad5155e598d6763d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40065 } health_report { overall_health: UNKNOWN } } }
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:07.335893 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.328358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.13.160.129:40065)
0925 05:15:07.328424 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.328426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.328534 (+ 108us) server_negotiation.cc:407] Connection header received
0925 05:15:07.328581 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.328582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.328591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.328647 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.329225 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.329714 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.330418 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.330627 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.330683 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.333048 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.333069 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.333070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.333077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.334601 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.335020 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.335025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.335025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.335080 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.335288 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.335289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.335290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.335448 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.335639 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.335689 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.335774 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260925 05:15:07.351780 14716 tablet.cc:2369] T 77cecc21735f4f85beccc9db1d3bc8f3 P c56cbebc4ad043e9ad5155e598d6763d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:07.359516 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.352052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.13.160.129:40065)
0925 05:15:07.352096 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.352097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.352264 (+ 167us) server_negotiation.cc:407] Connection header received
0925 05:15:07.352367 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.352368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.352383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.352448 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.353054 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.353531 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.354263 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.354463 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.354550 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.356956 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.356957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.356958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.356965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.358204 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.358587 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.358589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.358589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.358650 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.358852 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.358853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.358854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.359019 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.359133 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.359320 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.359406 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[14479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:07.386238 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:07.377603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.13.160.129:40065)
0925 05:15:07.377687 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:07.377689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:07.377904 (+ 215us) server_negotiation.cc:407] Connection header received
0925 05:15:07.377951 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:07.377952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:07.377966 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:07.378022 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:07.378812 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.379596 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.380695 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:07.381015 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:07.381098 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:07.384100 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:07.384102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:07.384102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:07.384111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:07.385344 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.385678 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.385679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.385680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.385733 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:07.385852 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:07.385853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:07.385853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:07.385940 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:07.386016 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:07.386108 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 05:15:07.386169 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 05:15:07.386830 14629 tablet_service.cc:2908] Rejecting scan request for tablet 77cecc21735f4f85beccc9db1d3bc8f3: Uninitialized: clean time has not yet been initialized
I20260925 05:15:07.387704 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14586
I20260925 05:15:07.393111 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14720
I20260925 05:15:07.398051 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14854
I20260925 05:15:07.402729 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 14492
2026-09-25T05:15:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2016 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[15024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[15024](info): set up 2 sockets
Sep 25 05:15:07 dist-test-slave-9tgz krb5kdc[15024](info): commencing operation
krb5kdc: starting...
W20260925 05:15:08.435391 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:08 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:08.553974 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:44065
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:08.636444 15037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:08.636734 15037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:08.636775 15037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:08.638049 15037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:08.638163 15037 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 05:15:08.638183 15037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:08.638208 15037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:08.638236 15037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:08.638266 15037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:08.639868 15037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:44065
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:08.640184 15037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:08.640431 15037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:08.645576 15043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:08.645579 15042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:08.645576 15045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:08.645601 15037 server_base.cc:1034] running on GCE node
I20260925 05:15:08.646009 15037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:08.646337 15037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:08.647511 15037 hybrid_clock.cc:648] HybridClock initialized: now 1790313308647502 us; error 40 us; skew 500 ppm
Sep 25 05:15:08 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:08.650076 15037 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:08.650537 15037 webserver.cc:460] Webserver started at http://127.13.160.190:32817/ using document root <none> and password file <none>
I20260925 05:15:08.650787 15037 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:08.650854 15037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:08.651036 15037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:08.652237 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "a8ece354bfd245748acbc944066da1aa"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "fc58c864293bcfff4c5cab00c969b627"
server_key_iv: "2f4cddc6ffb0a480dd9320fb52b365e2"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.652616 15037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "a8ece354bfd245748acbc944066da1aa"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "fc58c864293bcfff4c5cab00c969b627"
server_key_iv: "2f4cddc6ffb0a480dd9320fb52b365e2"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.653981 15037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:08.654732 15052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:08.654970 15037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:08.655083 15037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "a8ece354bfd245748acbc944066da1aa"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "fc58c864293bcfff4c5cab00c969b627"
server_key_iv: "2f4cddc6ffb0a480dd9320fb52b365e2"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.655176 15037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:08.674188 15037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:08.676762 15037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:08.676941 15037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:08.681746 15037 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:44065
I20260925 05:15:08.681829 15114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:44065 every 8 connection(s)
I20260925 05:15:08.682189 15037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:08.682982 15115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:08.685390 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Bootstrap starting.
I20260925 05:15:08.686029 15115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:08.686348 15115 log.cc:824] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:08.687299 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: No bootstrap required, opened a new log
I20260925 05:15:08.689137 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15037
I20260925 05:15:08.689267 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:08.689426 13954 external_mini_cluster.cc:1428] Setting key d672e24e0311e5d56676812ae3439c0d
I20260925 05:15:08.689627 15115 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } }
I20260925 05:15:08.689858 15115 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:08.689958 15115 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ece354bfd245748acbc944066da1aa, State: Initialized, Role: FOLLOWER
I20260925 05:15:08.690124 15115 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } }
I20260925 05:15:08.690238 15115 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:08.690297 15115 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:08.690356 15115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:08.691160 15115 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } }
I20260925 05:15:08.691289 15115 leader_election.cc:302] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ece354bfd245748acbc944066da1aa; no voters:
I20260925 05:15:08.691478 15115 leader_election.cc:288] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:08.691526 15120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:08.691676 15120 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [term 1 LEADER]: Becoming Leader. State: Replica: a8ece354bfd245748acbc944066da1aa, State: Running, Role: LEADER
I20260925 05:15:08.691802 15115 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:08.691828 15120 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } }
I20260925 05:15:08.692337 15122 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8ece354bfd245748acbc944066da1aa. Latest consensus state: current_term: 1 leader_uuid: "a8ece354bfd245748acbc944066da1aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } } }
I20260925 05:15:08.692342 15121 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8ece354bfd245748acbc944066da1aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ece354bfd245748acbc944066da1aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44065 } } }
I20260925 05:15:08.692459 15122 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:08.692467 15121 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:08.692658 15129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:08.693193 15129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:08.695051 15129 catalog_manager.cc:1347] Generated new cluster ID: 29e6da4a2f2142d492496f682b172f88
I20260925 05:15:08.695104 15129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:08 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:08.702414 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:08.690090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.13.160.190:44065)
0925 05:15:08.690314 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:08.690326 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:08.690379 (+ 53us) server_negotiation.cc:407] Connection header received
0925 05:15:08.690518 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:08.690524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:08.690572 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:08.690687 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:08.691553 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.693029 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.694236 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.694883 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.695020 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:08.697951 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:08.697957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:08.697957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:08.697970 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:08.700597 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.701071 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.701074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.701074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.701217 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.701449 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.701450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.701450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.701770 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:08.701913 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:08.702048 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 05:15:08.702186 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":141,"threads_started":1}
I20260925 05:15:08.715973 15129 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:08.716766 15129 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:08.725528 15129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Generated new TSK 0
I20260925 05:15:08.725791 15129 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:08.758294 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44065
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:08.841637 15141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:08.841962 15141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:08.841995 15141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:08.843369 15141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:08.843413 15141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:08.843434 15141 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:08.843453 15141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:08.844942 15141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:08.845223 15141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:08.845469 15141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:08.850417 15147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:08.850477 15149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:08.850414 15146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:08.850648 15141 server_base.cc:1034] running on GCE node
I20260925 05:15:08.850946 15141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:08.851251 15141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:08.852418 15141 hybrid_clock.cc:648] HybridClock initialized: now 1790313308852411 us; error 37 us; skew 500 ppm
Sep 25 05:15:08 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:08.855094 15141 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:08.855491 15141 webserver.cc:460] Webserver started at http://127.13.160.129:37663/ using document root <none> and password file <none>
I20260925 05:15:08.855695 15141 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:08.855737 15141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:08.855829 15141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:08.856842 15141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "d74e9af1e5cc450e96efa170e3dd557d"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "ded2925e572e1bfd10df0c65afffebbd"
server_key_iv: "64e818a7c6beb6068f8abfc788bd1c34"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.857146 15141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "d74e9af1e5cc450e96efa170e3dd557d"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "ded2925e572e1bfd10df0c65afffebbd"
server_key_iv: "64e818a7c6beb6068f8abfc788bd1c34"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.858521 15141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:08.859305 15156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:08.859475 15141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:08.859562 15141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "d74e9af1e5cc450e96efa170e3dd557d"
format_stamp: "Formatted at 2026-09-25 05:15:08 on dist-test-slave-9tgz"
server_key: "ded2925e572e1bfd10df0c65afffebbd"
server_key_iv: "64e818a7c6beb6068f8abfc788bd1c34"
server_key_version: "encryptionkey@0"
I20260925 05:15:08.859627 15141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:08.875602 15141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:08.877488 15141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:08.877604 15141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:08.878083 15141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:08.878123 15141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:08.878149 15141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:08.878199 15141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:08.885100 15141 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:37429
I20260925 05:15:08.885497 15141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:08 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:08.886174 15271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:37429 every 8 connection(s)
I20260925 05:15:08.888532 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:08.878640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:34345 (local address 127.13.160.190:44065)
0925 05:15:08.878717 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:08.878720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:08.878927 (+ 207us) server_negotiation.cc:407] Connection header received
0925 05:15:08.879500 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:08.879501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:08.879517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:08.879597 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:08.880508 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.881373 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.883056 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.883402 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.883493 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:08.886126 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:08.886128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:08.886128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:08.886138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:08.887408 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.887771 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.887771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.887772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.887820 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.888028 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.888028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.888029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.888135 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:08.888185 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:08.888403 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 05:15:08.888468 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:15:08.888747 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:08.878604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.129:34345)
0925 05:15:08.878774 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:08.878816 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:08.879355 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:08.879700 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:08.879704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:08.880101 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:08.880394 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:08.880400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.881512 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.881513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:08.882945 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:08.882948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.883503 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.883504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:08.883675 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:08.884286 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:08.884298 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:08.886041 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:08.887481 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:08.887487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:08.887488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:08.887704 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:08.887878 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:08.887879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:08.887879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:08.887975 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:08.888189 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:08.888193 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:08.888304 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 05:15:08.888386 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 05:15:08.888500 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260925 05:15:08.889302 15272 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44065
I20260925 05:15:08.889405 15272 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:08.889581 15272 heartbeater.cc:511] Master 127.13.160.190:44065 requested a full tablet report, sending...
I20260925 05:15:08.890105 15069 ts_manager.cc:194] Registered new tserver with Master: d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429)
I20260925 05:15:08.890797 15069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:34345
I20260925 05:15:08.893874 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15141
I20260925 05:15:08.893989 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:08.894220 13954 external_mini_cluster.cc:1428] Setting key f4f8b8747d0431d73af5264f85d5c197
I20260925 05:15:08.896528 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:08.890353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:46981 (local address 127.13.160.190:44065)
0925 05:15:08.890972 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:08.890973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:08.890982 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:08.891002 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:08.891003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:08.891013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:08.891067 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:08.891411 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.891991 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.892715 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.892966 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.893051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:08.893915 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:08.893916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:08.893916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:08.893924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:08.895392 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.895754 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.895755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.895756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.895810 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:08.896011 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:08.896011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:08.896012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:08.896124 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:08.896199 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:08.896352 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 05:15:08.896431 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594}
I20260925 05:15:08.896541 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:08.890295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.129:46981)
0925 05:15:08.890425 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:08.890438 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:08.890488 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:08.891086 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:08.891088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:08.891276 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:08.891331 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:08.891332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.892098 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.892098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:08.892606 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:08.892607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:08.893051 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:08.893052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:08.893151 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:08.893523 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:08.893527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:08.893824 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:08.895492 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:08.895492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:08.895493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:08.895666 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:08.895882 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:08.895883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:08.895883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:08.895944 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:08.896191 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:08.896192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:08.896240 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 05:15:08.896327 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 05:15:08.896433 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:08.919421 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44065
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:09.001904 15281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:09.002149 15281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:09.002193 15281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:09.003520 15281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:09.003593 15281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:09.003631 15281 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:09.003669 15281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:09.005122 15281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:09.005416 15281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:09.005688 15281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:09.011056 15289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:09.011044 15286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:09.011042 15287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:09.011226 15281 server_base.cc:1034] running on GCE node
I20260925 05:15:09.011523 15281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:09.011742 15281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:09.012919 15281 hybrid_clock.cc:648] HybridClock initialized: now 1790313309012911 us; error 39 us; skew 500 ppm
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:09.015399 15281 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:09.015885 15281 webserver.cc:460] Webserver started at http://127.13.160.130:38445/ using document root <none> and password file <none>
I20260925 05:15:09.016117 15281 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:09.016184 15281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:09.016335 15281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:09.017397 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "ee0239652cb74311b8461fd1905cea8f"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "4b15fcee2f11048a097e016002508541"
server_key_iv: "eac6d91a59573a0c61d74d4a8f15f1a2"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.017760 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "ee0239652cb74311b8461fd1905cea8f"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "4b15fcee2f11048a097e016002508541"
server_key_iv: "eac6d91a59573a0c61d74d4a8f15f1a2"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.019109 15281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:15:09.019790 15296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.019951 15281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:09.020056 15281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "ee0239652cb74311b8461fd1905cea8f"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "4b15fcee2f11048a097e016002508541"
server_key_iv: "eac6d91a59573a0c61d74d4a8f15f1a2"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.020135 15281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:09.031518 15281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:09.033344 15281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:09.033514 15281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:09.034029 15281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:09.034157 15281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.034214 15281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:09.034245 15281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.041101 15281 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:39555
I20260925 05:15:09.041183 15411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:39555 every 8 connection(s)
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:09.041524 15281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:09.044359 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15281
I20260925 05:15:09.044397 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.034548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:58035 (local address 127.13.160.190:44065)
0925 05:15:09.034615 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.034618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.034845 (+ 227us) server_negotiation.cc:407] Connection header received
0925 05:15:09.035432 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.035433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.035449 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.035528 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.036399 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.037254 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.038829 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.039180 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.039267 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.042025 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.042027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.042028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.042035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.043290 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.043654 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.043655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.043655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.043706 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.043864 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.043864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.043865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.043956 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.044001 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.044269 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.044334 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:09.044453 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:09.044633 13954 external_mini_cluster.cc:1428] Setting key 613fd6c4053b2ea023542b4a287aaf6b
I20260925 05:15:09.044647 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.034533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.130:58035)
0925 05:15:09.034702 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.034740 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.035325 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.035623 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.035627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.035984 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:09.036284 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.036290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.037355 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.037356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.038741 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.038744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.039267 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.039268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.039366 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.039944 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:09.039955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:09.041891 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:09.043362 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.043367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.043369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.043582 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.043746 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.043746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.043747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.043830 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.044009 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:09.044013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:09.044129 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.044250 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.044420 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260925 05:15:09.045166 15412 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44065
I20260925 05:15:09.045264 15412 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:09.045496 15412 heartbeater.cc:511] Master 127.13.160.190:44065 requested a full tablet report, sending...
I20260925 05:15:09.045853 15069 ts_manager.cc:194] Registered new tserver with Master: ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555)
I20260925 05:15:09.046356 15069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:58035
I20260925 05:15:09.052783 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.046203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.130:49685)
0925 05:15:09.046321 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.046330 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.046382 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.046508 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.046508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.047257 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:09.047310 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.047311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.048164 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.048165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.048659 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.048660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.049162 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.049163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.049218 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.049672 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:09.049677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:09.049957 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:09.051804 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.051805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.051805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.051962 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.052173 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.052174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.052174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.052222 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.052539 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:09.052539 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:09.052573 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.052637 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.052707 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260925 05:15:09.052878 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.046251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:49685 (local address 127.13.160.190:44065)
0925 05:15:09.046291 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.046293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.046368 (+ 75us) server_negotiation.cc:407] Connection header received
0925 05:15:09.046438 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.046439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.046449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.046503 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.047407 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.048069 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.048745 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.049088 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.049166 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.050047 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.050049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.050049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.050065 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.051719 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.052036 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.052037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.052037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.052107 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.052278 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.052279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.052279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.052465 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.052540 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.052689 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.052764 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:09.070276 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44065
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:09.154162 15421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:09.154388 15421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:09.154415 15421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:09.155699 15421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:09.155746 15421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:09.155768 15421 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:09.155824 15421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:09.157265 15421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:09.157509 15421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:09.157765 15421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:09.162720 15427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:09.162715 15429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:09.162715 15426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:09.162958 15421 server_base.cc:1034] running on GCE node
I20260925 05:15:09.163160 15421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:09.163439 15421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:09.164604 15421 hybrid_clock.cc:648] HybridClock initialized: now 1790313309164595 us; error 41 us; skew 500 ppm
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:09.167068 15421 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:09.167555 15421 webserver.cc:460] Webserver started at http://127.13.160.131:45431/ using document root <none> and password file <none>
I20260925 05:15:09.167800 15421 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:09.167867 15421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:09.168002 15421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:09.169091 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "45624b02321b4436b449f062e38bb715"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "1555e5685b6a68ba868e70481aaf34cc"
server_key_iv: "ea717183ac3d1a996d53119bf0320557"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.169456 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "45624b02321b4436b449f062e38bb715"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "1555e5685b6a68ba868e70481aaf34cc"
server_key_iv: "ea717183ac3d1a996d53119bf0320557"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.170795 15421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:09.171550 15436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.171725 15421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:09.171821 15421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "45624b02321b4436b449f062e38bb715"
format_stamp: "Formatted at 2026-09-25 05:15:09 on dist-test-slave-9tgz"
server_key: "1555e5685b6a68ba868e70481aaf34cc"
server_key_iv: "ea717183ac3d1a996d53119bf0320557"
server_key_version: "encryptionkey@0"
I20260925 05:15:09.171907 15421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:09.179869 15421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:09.181697 15421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:09.181856 15421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:09.182407 15421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:09.182466 15421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.182544 15421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:09.182576 15421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:09.189910 15421 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:37941
I20260925 05:15:09.189960 15551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:37941 every 8 connection(s)
I20260925 05:15:09.190389 15421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:09.192202 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.182865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:32949 (local address 127.13.160.190:44065)
0925 05:15:09.182956 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.182959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.183191 (+ 232us) server_negotiation.cc:407] Connection header received
0925 05:15:09.183696 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.183698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.183711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.183768 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.184409 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.185026 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.186089 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.186300 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.186385 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.189149 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.189150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.189151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.189158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.191004 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.191411 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.191412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.191412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.191474 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.191718 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.191718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.191719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.191822 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.191872 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.192077 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.192140 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 05:15:09.192378 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.182853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.131:32949)
0925 05:15:09.183058 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.183097 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.183587 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.183841 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.183845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.184136 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:09.184315 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.184319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.185111 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.185112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.185971 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.185973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.186394 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.186395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.186562 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.187147 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:09.187156 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:09.189040 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:09.191071 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.191075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.191077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.191283 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.191538 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.191539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.191539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.191656 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.191862 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:09.191864 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:09.191970 (+ 106us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.192047 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.192166 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":130,"threads_started":1}
I20260925 05:15:09.192914 15552 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44065
I20260925 05:15:09.192996 15552 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:09.193194 15552 heartbeater.cc:511] Master 127.13.160.190:44065 requested a full tablet report, sending...
I20260925 05:15:09.193645 15068 ts_manager.cc:194] Registered new tserver with Master: 45624b02321b4436b449f062e38bb715 (127.13.160.131:37941)
I20260925 05:15:09.194033 15068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:32949
I20260925 05:15:09.195158 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15421
I20260925 05:15:09.195250 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:09.195430 13954 external_mini_cluster.cc:1428] Setting key 3f7fcf4271404290aca45a6230851ee6
I20260925 05:15:09.196344 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:09.199965 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.194087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:40011 (local address 127.13.160.190:44065)
0925 05:15:09.194157 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.194158 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.194169 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:09.194258 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.194259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.194268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.194315 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.194645 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.195314 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.196189 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.196467 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.196543 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.197394 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.197396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.197396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.197403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.198818 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.199210 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.199211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.199212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.199266 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.199501 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.199503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.199503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.199614 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.199675 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.199844 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.199905 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 05:15:09.200103 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.194007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.13.160.131:40011)
0925 05:15:09.194119 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.194130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.194182 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.194318 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.194318 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.194497 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:09.194561 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.194563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.195401 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.195401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.196095 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.196097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.196542 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.196543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.196608 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.197037 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:09.197041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:09.197307 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:09.198931 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.198932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.198932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.199131 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.199342 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.199343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.199343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.199408 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.199688 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:09.199689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:09.199742 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.199816 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.199929 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:09.220214 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.213412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.13.160.190:44065)
0925 05:15:09.213480 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.213482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.213492 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:09.213565 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.213565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.213575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.213627 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.214143 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.214626 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.215454 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.215634 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.215685 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.217906 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.217907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.217908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.217914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.219288 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.219601 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.219602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.219602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.219653 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.219813 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.219814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.219814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.219914 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.219990 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.220090 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.220146 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 05:15:09.221817 15068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:09.223587 15068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:09.229930 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.225889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37941 (local address 127.0.0.1:55394)
0925 05:15:09.226075 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.226116 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.226183 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.226462 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.226465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.226468 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.226544 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.226547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.227627 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.227629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.228656 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.228658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.229605 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.229606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.229692 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.229704 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.229778 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.229836 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
I20260925 05:15:09.229938 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.225990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.13.160.131:37941)
0925 05:15:09.226219 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.226224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.226238 (+ 14us) server_negotiation.cc:407] Connection header received
0925 05:15:09.226280 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.226282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.226302 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.226382 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.226622 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.227541 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.228772 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.229496 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.229606 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.229671 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.229797 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.229836 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":113,"threads_started":1}
I20260925 05:15:09.230384 15569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.226281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39555 (local address 127.0.0.1:38254)
0925 05:15:09.226460 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.226470 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.226513 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.226922 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.226923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.226924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.226958 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.226959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.228247 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.228247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.229159 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.229161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.230150 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.230151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.230211 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.230215 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.230269 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.230310 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260925 05:15:09.230669 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.226379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.13.160.130:39555)
0925 05:15:09.226682 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.226688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.226697 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:09.226734 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.226736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.226751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.226830 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.227048 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.228158 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.229243 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.229989 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.230062 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.230117 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.230523 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.230572 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":76,"threads_started":1}
I20260925 05:15:09.230883 15486 tablet_service.cc:1467] Processing CreateTablet for tablet c24a17f99df046cb8bb9b4645532c6fb (DEFAULT_TABLE table=test-table [id=8131d64cfb784a2fa11b8603f2fa5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.231158 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.226461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.13.160.129:37429)
0925 05:15:09.226593 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.226598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.226609 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:09.226647 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.226650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.226664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.226995 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.227055 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.228145 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.229465 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.230418 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.230645 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.230705 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.231001 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.231056 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260925 05:15:09.230870 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.226137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37429 (local address 127.0.0.1:33004)
0925 05:15:09.226215 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.226225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.226266 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.226777 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.226778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.226780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.226832 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.226834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.228241 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.228242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.229370 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.229373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.230644 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.230645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.230704 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.230708 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.230757 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.230795 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":30,"thread_start_us":49,"threads_started":1}
I20260925 05:15:09.231307 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a17f99df046cb8bb9b4645532c6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.231359 15346 tablet_service.cc:1467] Processing CreateTablet for tablet c24a17f99df046cb8bb9b4645532c6fb (DEFAULT_TABLE table=test-table [id=8131d64cfb784a2fa11b8603f2fa5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.231662 15346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a17f99df046cb8bb9b4645532c6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.231777 15206 tablet_service.cc:1467] Processing CreateTablet for tablet c24a17f99df046cb8bb9b4645532c6fb (DEFAULT_TABLE table=test-table [id=8131d64cfb784a2fa11b8603f2fa5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.232039 15206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a17f99df046cb8bb9b4645532c6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.234750 15572 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Bootstrap starting.
I20260925 05:15:09.235064 15573 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Bootstrap starting.
I20260925 05:15:09.235481 15574 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Bootstrap starting.
I20260925 05:15:09.235706 15572 tablet_bootstrap.cc:654] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.235749 15573 tablet_bootstrap.cc:654] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.236019 15572 log.cc:824] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:09.236085 15573 log.cc:824] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:09.236110 15574 tablet_bootstrap.cc:654] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.236415 15574 log.cc:824] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:09.236799 15572 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: No bootstrap required, opened a new log
I20260925 05:15:09.236872 15572 ts_tablet_manager.cc:1397] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:09.237134 15574 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: No bootstrap required, opened a new log
I20260925 05:15:09.237212 15574 ts_tablet_manager.cc:1397] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:09.238303 15572 raft_consensus.cc:348] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.238436 15572 raft_consensus.cc:374] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.238458 15572 raft_consensus.cc:729] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45624b02321b4436b449f062e38bb715, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.238590 15572 consensus_queue.cc:260] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.238622 15574 raft_consensus.cc:348] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.238843 15572 ts_tablet_manager.cc:1428] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:09.238801 15574 raft_consensus.cc:374] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.238971 15574 raft_consensus.cc:729] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0239652cb74311b8461fd1905cea8f, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.238866 15552 heartbeater.cc:503] Master 127.13.160.190:44065 was elected leader, sending a full tablet report...
I20260925 05:15:09.239143 15574 consensus_queue.cc:260] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.239395 15574 ts_tablet_manager.cc:1428] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:15:09.239549 15412 heartbeater.cc:503] Master 127.13.160.190:44065 was elected leader, sending a full tablet report...
I20260925 05:15:09.239764 15573 tablet_bootstrap.cc:492] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: No bootstrap required, opened a new log
I20260925 05:15:09.239835 15573 ts_tablet_manager.cc:1397] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 05:15:09.241422 15573 raft_consensus.cc:348] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.241580 15573 raft_consensus.cc:374] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.241626 15573 raft_consensus.cc:729] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74e9af1e5cc450e96efa170e3dd557d, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.241735 15573 consensus_queue.cc:260] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.241981 15573 ts_tablet_manager.cc:1428] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:09.242054 15272 heartbeater.cc:503] Master 127.13.160.190:44065 was elected leader, sending a full tablet report...
W20260925 05:15:09.292187 15413 tablet.cc:2369] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:15:09.386224 15273 tablet.cc:2369] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:15:09.440986 15553 tablet.cc:2369] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:09.480007 15579 raft_consensus.cc:482] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:09.480147 15579 raft_consensus.cc:504] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.480460 15579 leader_election.cc:288] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45624b02321b4436b449f062e38bb715 (127.13.160.131:37941), d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429)
I20260925 05:15:09.483465 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.480836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37429 (local address 127.13.160.130:55457)
0925 05:15:09.480993 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.481003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.481061 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.481217 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.481218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.481220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.481291 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.481292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.481920 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.481921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.482647 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.482648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.483193 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.483195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.483281 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.483286 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.483339 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.483368 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260925 05:15:09.483479 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.480923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:55457 (local address 127.13.160.129:37429)
0925 05:15:09.480967 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.480969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.481070 (+ 101us) server_negotiation.cc:407] Connection header received
0925 05:15:09.481107 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.481109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.481118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.481183 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.481363 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.481840 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.482743 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.483118 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.483172 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.483179 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.483361 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.483401 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:09.483583 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.480839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:51913 (local address 127.13.160.131:37941)
0925 05:15:09.480888 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.480890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.480904 (+ 14us) server_negotiation.cc:407] Connection header received
0925 05:15:09.480942 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.480943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.480965 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.481041 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.481241 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.481775 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.482741 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.483199 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.483273 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.483280 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.483469 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.483512 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:15:09.483579 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.480702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37941 (local address 127.13.160.130:51913)
0925 05:15:09.480786 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.480798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.480891 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.481072 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.481073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.481075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.481162 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.481164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.481857 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.481858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.482644 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.482645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.483286 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.483287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.483385 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.483388 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.483445 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.483482 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 05:15:09.484068 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a17f99df046cb8bb9b4645532c6fb" candidate_uuid: "ee0239652cb74311b8461fd1905cea8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d" is_pre_election: true
I20260925 05:15:09.484066 15506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a17f99df046cb8bb9b4645532c6fb" candidate_uuid: "ee0239652cb74311b8461fd1905cea8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45624b02321b4436b449f062e38bb715" is_pre_election: true
I20260925 05:15:09.484242 15226 raft_consensus.cc:2454] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee0239652cb74311b8461fd1905cea8f in term 0.
I20260925 05:15:09.484242 15506 raft_consensus.cc:2454] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee0239652cb74311b8461fd1905cea8f in term 0.
I20260925 05:15:09.484510 15297 leader_election.cc:302] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, ee0239652cb74311b8461fd1905cea8f; no voters:
I20260925 05:15:09.484645 15579 raft_consensus.cc:2781] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:09.484730 15579 raft_consensus.cc:482] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:09.484763 15579 raft_consensus.cc:3037] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.485700 15579 raft_consensus.cc:504] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.485842 15579 leader_election.cc:288] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [CANDIDATE]: Term 1 election: Requested vote from peers 45624b02321b4436b449f062e38bb715 (127.13.160.131:37941), d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429)
I20260925 05:15:09.485988 15506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a17f99df046cb8bb9b4645532c6fb" candidate_uuid: "ee0239652cb74311b8461fd1905cea8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45624b02321b4436b449f062e38bb715"
I20260925 05:15:09.486064 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a17f99df046cb8bb9b4645532c6fb" candidate_uuid: "ee0239652cb74311b8461fd1905cea8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d"
I20260925 05:15:09.486090 15506 raft_consensus.cc:3037] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.486138 15226 raft_consensus.cc:3037] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.486989 15506 raft_consensus.cc:2454] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee0239652cb74311b8461fd1905cea8f in term 1.
I20260925 05:15:09.486989 15226 raft_consensus.cc:2454] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee0239652cb74311b8461fd1905cea8f in term 1.
I20260925 05:15:09.487155 15297 leader_election.cc:302] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, ee0239652cb74311b8461fd1905cea8f; no voters:
I20260925 05:15:09.487244 15579 raft_consensus.cc:2781] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:09.487424 15579 raft_consensus.cc:686] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 1 LEADER]: Becoming Leader. State: Replica: ee0239652cb74311b8461fd1905cea8f, State: Running, Role: LEADER
I20260925 05:15:09.487520 15579 consensus_queue.cc:237] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } }
I20260925 05:15:09.488348 15068 catalog_manager.cc:5495] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ee0239652cb74311b8461fd1905cea8f (127.13.160.130). New cstate: current_term: 1 leader_uuid: "ee0239652cb74311b8461fd1905cea8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } health_report { overall_health: HEALTHY } } }
I20260925 05:15:09.515039 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.512358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.13.160.130:39555)
0925 05:15:09.512418 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.512420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.512460 (+ 40us) server_negotiation.cc:407] Connection header received
0925 05:15:09.512628 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.512634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.512652 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.512742 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:09.512891 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.513364 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.514165 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.514357 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.514432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.514650 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.514751 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.514842 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.514884 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.514932 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:15:09.516897 15226 raft_consensus.cc:1261] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Refusing update from remote peer ee0239652cb74311b8461fd1905cea8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.516938 15506 raft_consensus.cc:1261] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Refusing update from remote peer ee0239652cb74311b8461fd1905cea8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.517253 15579 consensus_queue.cc:1035] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.517418 15582 consensus_queue.cc:1035] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.520021 15587 mvcc.cc:205] Tried to move back new op lower bound from 7333123315778154496 to 7333123315661398016. Current Snapshot: MvccSnapshot[applied={T|T < 7333123315778154496}]
I20260925 05:15:09.523020 15069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:09.523128 15069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:09.523841 15069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Sending DeleteTablet for 3 replicas of tablet c24a17f99df046cb8bb9b4645532c6fb
I20260925 05:15:09.524183 15206 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a17f99df046cb8bb9b4645532c6fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:33004
I20260925 05:15:09.524211 15486 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a17f99df046cb8bb9b4645532c6fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55394
I20260925 05:15:09.524358 15069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:09.524508 15069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:09.524482 15346 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a17f99df046cb8bb9b4645532c6fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:38254
I20260925 05:15:09.524896 15596 tablet_replica.cc:330] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: stopping tablet replica
I20260925 05:15:09.525005 15596 raft_consensus.cc:2229] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:09.525133 15596 raft_consensus.cc:2258] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.525317 15597 tablet_replica.cc:330] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: stopping tablet replica
I20260925 05:15:09.525422 15597 raft_consensus.cc:2229] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:09.525547 15597 raft_consensus.cc:2258] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.525720 15595 tablet_replica.cc:330] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: stopping tablet replica
I20260925 05:15:09.525918 15346 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2d12da19647eabeb7b93ae8968fda (DEFAULT_TABLE table=test-table [id=f8307b2996764eb08c1cb9886918bca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.525971 15206 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2d12da19647eabeb7b93ae8968fda (DEFAULT_TABLE table=test-table [id=f8307b2996764eb08c1cb9886918bca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.526043 15346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2d12da19647eabeb7b93ae8968fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.526085 15206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2d12da19647eabeb7b93ae8968fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.527019 15486 tablet_service.cc:1467] Processing CreateTablet for tablet e4a2d12da19647eabeb7b93ae8968fda (DEFAULT_TABLE table=test-table [id=f8307b2996764eb08c1cb9886918bca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:09.527113 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a2d12da19647eabeb7b93ae8968fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.527458 15573 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Bootstrap starting.
I20260925 05:15:09.527493 15574 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Bootstrap starting.
I20260925 05:15:09.527849 15573 tablet_bootstrap.cc:654] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.527849 15574 tablet_bootstrap.cc:654] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.528228 15572 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Bootstrap starting.
I20260925 05:15:09.528489 15574 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: No bootstrap required, opened a new log
I20260925 05:15:09.528554 15574 ts_tablet_manager.cc:1397] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:09.528633 15572 tablet_bootstrap.cc:654] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.528714 15574 raft_consensus.cc:348] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.528795 15574 raft_consensus.cc:374] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.528820 15574 raft_consensus.cc:729] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0239652cb74311b8461fd1905cea8f, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.528868 15574 consensus_queue.cc:260] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.528977 15574 ts_tablet_manager.cc:1428] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:09.529531 15597 ts_tablet_manager.cc:1905] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.530745 15596 ts_tablet_manager.cc:1905] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.531196 15595 raft_consensus.cc:2229] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:09.531329 15595 raft_consensus.cc:2258] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.531760 15572 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: No bootstrap required, opened a new log
I20260925 05:15:09.531810 15572 ts_tablet_manager.cc:1397] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 05:15:09.531890 15595 ts_tablet_manager.cc:1905] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.531945 15572 raft_consensus.cc:348] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.532018 15572 raft_consensus.cc:374] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.532040 15572 raft_consensus.cc:729] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45624b02321b4436b449f062e38bb715, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.532099 15572 consensus_queue.cc:260] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.532181 15572 ts_tablet_manager.cc:1428] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.532871 15597 ts_tablet_manager.cc:1918] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:09.532949 15597 log.cc:1196] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/c24a17f99df046cb8bb9b4645532c6fb
I20260925 05:15:09.533215 15597 ts_tablet_manager.cc:1939] T c24a17f99df046cb8bb9b4645532c6fb P ee0239652cb74311b8461fd1905cea8f: Deleting consensus metadata
I20260925 05:15:09.533466 15595 ts_tablet_manager.cc:1918] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:09.533529 15595 log.cc:1196] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/c24a17f99df046cb8bb9b4645532c6fb
I20260925 05:15:09.533696 15596 ts_tablet_manager.cc:1918] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:09.533762 15596 log.cc:1196] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/c24a17f99df046cb8bb9b4645532c6fb
I20260925 05:15:09.533785 15595 ts_tablet_manager.cc:1939] T c24a17f99df046cb8bb9b4645532c6fb P d74e9af1e5cc450e96efa170e3dd557d: Deleting consensus metadata
I20260925 05:15:09.534005 15596 ts_tablet_manager.cc:1939] T c24a17f99df046cb8bb9b4645532c6fb P 45624b02321b4436b449f062e38bb715: Deleting consensus metadata
I20260925 05:15:09.534178 15053 catalog_manager.cc:4841] TS d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429): tablet c24a17f99df046cb8bb9b4645532c6fb (table test-table [id=8131d64cfb784a2fa11b8603f2fa5854]) successfully deleted
I20260925 05:15:09.534399 15053 catalog_manager.cc:4841] TS 45624b02321b4436b449f062e38bb715 (127.13.160.131:37941): tablet c24a17f99df046cb8bb9b4645532c6fb (table test-table [id=8131d64cfb784a2fa11b8603f2fa5854]) successfully deleted
I20260925 05:15:09.534664 15053 catalog_manager.cc:4841] TS ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555): tablet c24a17f99df046cb8bb9b4645532c6fb (table test-table [id=8131d64cfb784a2fa11b8603f2fa5854]) successfully deleted
I20260925 05:15:09.537351 15578 raft_consensus.cc:482] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:09.537431 15578 raft_consensus.cc:504] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.537708 15578 leader_election.cc:288] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429), ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555)
I20260925 05:15:09.538129 15573 tablet_bootstrap.cc:492] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: No bootstrap required, opened a new log
I20260925 05:15:09.538182 15573 ts_tablet_manager.cc:1397] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260925 05:15:09.538333 15573 raft_consensus.cc:348] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.538412 15573 raft_consensus.cc:374] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.538435 15573 raft_consensus.cc:729] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74e9af1e5cc450e96efa170e3dd557d, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.538479 15573 consensus_queue.cc:260] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.538595 15573 ts_tablet_manager.cc:1428] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:09.541177 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.538077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:41165 (local address 127.13.160.130:39555)
0925 05:15:09.538114 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.538116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.538213 (+ 97us) server_negotiation.cc:407] Connection header received
0925 05:15:09.538313 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.538313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.538323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.538352 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.538578 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.539283 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.540433 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.540810 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.540890 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.540899 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.541074 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.541110 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:15:09.541198 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.537955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39555 (local address 127.13.160.131:41165)
0925 05:15:09.538152 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.538162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.538239 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.538426 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.538427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.538428 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.538501 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.538503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.539385 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.539386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.540301 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.540303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.540915 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.540916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.540991 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.540994 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.541066 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.541110 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":47,"threads_started":1}
I20260925 05:15:09.541411 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.537846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37429 (local address 127.13.160.131:58981)
0925 05:15:09.537934 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.537947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.537992 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.538185 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.538186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.538187 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:09.538244 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.538245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.539187 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.539188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.540398 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.540400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.541155 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.541156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.541248 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.541251 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.541300 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.541345 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 05:15:09.541585 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.537889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:58981 (local address 127.13.160.129:37429)
0925 05:15:09.538063 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.538065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.538071 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:09.538092 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.538093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.538101 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.538144 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:09.538309 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.539094 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.540497 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.541050 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.541117 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.541125 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.541481 (+ 356us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.541530 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260925 05:15:09.541762 15366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2d12da19647eabeb7b93ae8968fda" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0239652cb74311b8461fd1905cea8f" is_pre_election: true
I20260925 05:15:09.541877 15366 raft_consensus.cc:2454] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 0.
I20260925 05:15:09.541975 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2d12da19647eabeb7b93ae8968fda" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d" is_pre_election: true
I20260925 05:15:09.542042 15226 raft_consensus.cc:2454] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 0.
I20260925 05:15:09.542029 15437 leader_election.cc:302] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, ee0239652cb74311b8461fd1905cea8f; no voters:
I20260925 05:15:09.542220 15578 raft_consensus.cc:2781] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:09.542273 15578 raft_consensus.cc:482] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:09.542307 15578 raft_consensus.cc:3037] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.543022 15578 raft_consensus.cc:504] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.543145 15578 leader_election.cc:288] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 election: Requested vote from peers d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429), ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555)
I20260925 05:15:09.543325 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2d12da19647eabeb7b93ae8968fda" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d"
I20260925 05:15:09.543398 15226 raft_consensus.cc:3037] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.543349 15366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a2d12da19647eabeb7b93ae8968fda" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0239652cb74311b8461fd1905cea8f"
I20260925 05:15:09.543434 15366 raft_consensus.cc:3037] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.544251 15366 raft_consensus.cc:2454] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 1.
I20260925 05:15:09.544404 15437 leader_election.cc:302] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, ee0239652cb74311b8461fd1905cea8f; no voters:
I20260925 05:15:09.544478 15226 raft_consensus.cc:2454] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 1.
I20260925 05:15:09.544483 15578 raft_consensus.cc:2781] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:09.544667 15578 raft_consensus.cc:686] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 1 LEADER]: Becoming Leader. State: Replica: 45624b02321b4436b449f062e38bb715, State: Running, Role: LEADER
I20260925 05:15:09.544772 15578 consensus_queue.cc:237] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.545450 15069 catalog_manager.cc:5495] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45624b02321b4436b449f062e38bb715 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } health_report { overall_health: HEALTHY } } }
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:09.575271 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.567727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.13.160.190:44065)
0925 05:15:09.567800 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.567803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.567918 (+ 115us) server_negotiation.cc:407] Connection header received
0925 05:15:09.567991 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.567991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.568002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.568077 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.568664 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.569142 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.570174 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.570344 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.570389 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.572648 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.572650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.572650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.572657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.574049 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.574458 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.574459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.574459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.574516 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.574720 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.574721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.574722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.574830 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.574924 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.575078 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.575162 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 05:15:09.575340 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.567673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44065 (local address 127.0.0.1:47222)
0925 05:15:09.567841 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.567858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.567938 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.568113 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.568114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.568506 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:09.568575 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.568578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.569215 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.569216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.570078 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.570082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.570414 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.570414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.570476 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.571090 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:09.571095 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:09.572557 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:09.574145 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.574153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.574156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.574371 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.574589 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:09.574590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:09.574591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:09.574653 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:09.574916 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:09.574923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:09.574967 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.575057 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.575217 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":31,"thread_start_us":78,"threads_started":1}
I20260925 05:15:09.577955 15069 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:47222:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 05:15:09.578163 15069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:09.579589 15486 tablet_service.cc:1467] Processing CreateTablet for tablet 98b400f2db0d44399af15f86517c6235 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7cee13285b964d8cb9fc80dace249842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:09.579648 15206 tablet_service.cc:1467] Processing CreateTablet for tablet 98b400f2db0d44399af15f86517c6235 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7cee13285b964d8cb9fc80dace249842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:09.579717 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b400f2db0d44399af15f86517c6235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.579793 15206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b400f2db0d44399af15f86517c6235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.581243 15573 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: Bootstrap starting.
I20260925 05:15:09.581300 15346 tablet_service.cc:1467] Processing CreateTablet for tablet 98b400f2db0d44399af15f86517c6235 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7cee13285b964d8cb9fc80dace249842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:09.581425 15346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b400f2db0d44399af15f86517c6235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:09.581610 15573 tablet_bootstrap.cc:654] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.582137 15573 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: No bootstrap required, opened a new log
I20260925 05:15:09.582168 15573 ts_tablet_manager.cc:1397] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:09.582271 15573 raft_consensus.cc:348] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.582317 15573 raft_consensus.cc:374] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.582330 15573 raft_consensus.cc:729] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74e9af1e5cc450e96efa170e3dd557d, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.582391 15573 consensus_queue.cc:260] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.582502 15573 ts_tablet_manager.cc:1428] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:09.582554 15580 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.582638 15580 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.582718 15572 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: Bootstrap starting.
I20260925 05:15:09.583132 15572 tablet_bootstrap.cc:654] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.583453 15574 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: Bootstrap starting.
I20260925 05:15:09.583778 15574 tablet_bootstrap.cc:654] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:09.584391 15574 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: No bootstrap required, opened a new log
I20260925 05:15:09.584431 15574 ts_tablet_manager.cc:1397] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:09.584579 15574 raft_consensus.cc:348] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.584645 15574 raft_consensus.cc:374] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.584658 15574 raft_consensus.cc:729] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0239652cb74311b8461fd1905cea8f, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.584641 15572 tablet_bootstrap.cc:492] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: No bootstrap required, opened a new log
I20260925 05:15:09.584685 15572 ts_tablet_manager.cc:1397] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:09.584688 15574 consensus_queue.cc:260] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.584779 15574 ts_tablet_manager.cc:1428] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.584811 15591 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.584817 15572 raft_consensus.cc:348] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.584877 15572 raft_consensus.cc:374] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:09.584877 15591 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.584898 15572 raft_consensus.cc:729] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45624b02321b4436b449f062e38bb715, State: Initialized, Role: FOLLOWER
I20260925 05:15:09.584946 15572 consensus_queue.cc:260] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.585035 15572 ts_tablet_manager.cc:1428] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:09.585124 15578 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.585224 15578 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.625216 15578 raft_consensus.cc:482] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:09.625304 15578 raft_consensus.cc:504] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.625466 15578 leader_election.cc:288] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429), ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555)
I20260925 05:15:09.625752 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b400f2db0d44399af15f86517c6235" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d" is_pre_election: true
I20260925 05:15:09.625865 15226 raft_consensus.cc:2454] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 0.
I20260925 05:15:09.625787 15366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b400f2db0d44399af15f86517c6235" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0239652cb74311b8461fd1905cea8f" is_pre_election: true
I20260925 05:15:09.626118 15366 raft_consensus.cc:2454] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 0.
I20260925 05:15:09.626119 15437 leader_election.cc:302] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, d74e9af1e5cc450e96efa170e3dd557d; no voters:
I20260925 05:15:09.626211 15578 raft_consensus.cc:2781] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:09.626243 15578 raft_consensus.cc:482] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:09.626263 15578 raft_consensus.cc:3037] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.627355 15578 raft_consensus.cc:504] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.627496 15578 leader_election.cc:288] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 election: Requested vote from peers d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429), ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555)
I20260925 05:15:09.627707 15226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b400f2db0d44399af15f86517c6235" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74e9af1e5cc450e96efa170e3dd557d"
I20260925 05:15:09.627784 15226 raft_consensus.cc:3037] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.627761 15366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b400f2db0d44399af15f86517c6235" candidate_uuid: "45624b02321b4436b449f062e38bb715" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0239652cb74311b8461fd1905cea8f"
I20260925 05:15:09.627831 15366 raft_consensus.cc:3037] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:09.628440 15226 raft_consensus.cc:2454] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 1.
I20260925 05:15:09.628646 15437 leader_election.cc:302] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45624b02321b4436b449f062e38bb715, d74e9af1e5cc450e96efa170e3dd557d; no voters:
I20260925 05:15:09.628675 15366 raft_consensus.cc:2454] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45624b02321b4436b449f062e38bb715 in term 1.
I20260925 05:15:09.628736 15578 raft_consensus.cc:2781] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:09.628782 15578 raft_consensus.cc:686] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [term 1 LEADER]: Becoming Leader. State: Replica: 45624b02321b4436b449f062e38bb715, State: Running, Role: LEADER
I20260925 05:15:09.628837 15578 consensus_queue.cc:237] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } }
I20260925 05:15:09.629179 15578 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: TxnStatusTablet state changed. Reason: New leader 45624b02321b4436b449f062e38bb715. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.629257 15578 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:09.629452 15614 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 05:15:09.629556 15614 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 05:15:09.629528 15069 catalog_manager.cc:5495] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45624b02321b4436b449f062e38bb715 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } health_report { overall_health: HEALTHY } } }
I20260925 05:15:09.636051 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.632952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.13.160.131:37941)
0925 05:15:09.632992 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.632994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.633000 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:09.633032 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.633033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.633052 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.633116 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:09.633295 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.633990 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.635149 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.635433 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.635503 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.635702 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.635811 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.635876 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.635953 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.635974 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:09.636092 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.632730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37941 (local address 127.0.0.1:55404)
0925 05:15:09.632810 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.632822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.632867 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.633153 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.633154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.633156 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:09.633210 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.633213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.634099 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.634100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.635067 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.635070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.635515 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.635517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.635589 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.635607 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.635881 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.635890 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.635949 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.635998 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 05:15:09.637220 15226 raft_consensus.cc:1261] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Refusing update from remote peer 45624b02321b4436b449f062e38bb715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.637555 15578 consensus_queue.cc:1035] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.637661 15366 raft_consensus.cc:1261] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Refusing update from remote peer 45624b02321b4436b449f062e38bb715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.637924 15578 consensus_queue.cc:1035] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.638191 15580 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: TxnStatusTablet state changed. Reason: New leader 45624b02321b4436b449f062e38bb715. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.638280 15580 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.638831 15591 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: TxnStatusTablet state changed. Reason: New leader 45624b02321b4436b449f062e38bb715. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.638944 15591 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.639652 15602 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.639734 15602 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:09.639933 15585 mvcc.cc:205] Tried to move back new op lower bound from 7333123316272148480 to 7333123316240154624. Current Snapshot: MvccSnapshot[applied={T|T < 7333123316272148480 or (T in {7333123316272148480})}]
I20260925 05:15:09.640033 15591 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.640106 15591 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P ee0239652cb74311b8461fd1905cea8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.641170 15602 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.641251 15602 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P 45624b02321b4436b449f062e38bb715: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:09.641598 15586 mvcc.cc:205] Tried to move back new op lower bound from 7333123316272148480 to 7333123316240154624. Current Snapshot: MvccSnapshot[applied={T|T < 7333123316272148480}]
I20260925 05:15:09.641705 15580 tablet_replica.cc:439] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45624b02321b4436b449f062e38bb715" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 } } peers { permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 } } peers { permanent_uuid: "45624b02321b4436b449f062e38bb715" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37941 } } }
I20260925 05:15:09.641768 15580 tablet_replica.cc:442] T 98b400f2db0d44399af15f86517c6235 P d74e9af1e5cc450e96efa170e3dd557d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:09.645072 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.642097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.13.160.131:37941)
0925 05:15:09.642159 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.642161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.642167 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:09.642185 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.642186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.642196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.642243 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:09.642433 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.643258 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.644279 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.644588 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.644659 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.644812 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.644846 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.644895 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.644952 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.644970 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:15:09.650763 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.647884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37941 (local address 127.13.160.131:37131)
0925 05:15:09.647941 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:09.647953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:09.648000 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:09.648148 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:09.648148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:09.648149 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:09.648199 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.648200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.649026 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.649027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.649937 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:09.649939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.650380 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.650380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:09.650436 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.650451 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.650606 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.650607 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:09.650640 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 05:15:09.650678 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 05:15:09.651199 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.647939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:37131 (local address 127.13.160.131:37941)
0925 05:15:09.647979 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.647981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.647991 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:09.648065 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.648066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.648075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.648117 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:09.648253 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.648955 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.650029 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.650307 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.650357 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.650527 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:09.650555 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:09.650591 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.651125 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.651141 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:15:09.653704 15366 raft_consensus.cc:1261] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Refusing update from remote peer 45624b02321b4436b449f062e38bb715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.653735 15226 raft_consensus.cc:1261] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Refusing update from remote peer 45624b02321b4436b449f062e38bb715: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:09.653942 15578 consensus_queue.cc:1035] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74e9af1e5cc450e96efa170e3dd557d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.654106 15621 consensus_queue.cc:1035] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0239652cb74311b8461fd1905cea8f" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:09.669416 15068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:09.669533 15068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:09.670157 15068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8ece354bfd245748acbc944066da1aa: Sending DeleteTablet for 3 replicas of tablet e4a2d12da19647eabeb7b93ae8968fda
I20260925 05:15:09.670462 15206 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a2d12da19647eabeb7b93ae8968fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:33004
I20260925 05:15:09.670495 15346 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a2d12da19647eabeb7b93ae8968fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:38254
I20260925 05:15:09.670593 15597 tablet_replica.cc:330] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: stopping tablet replica
I20260925 05:15:09.670601 15595 tablet_replica.cc:330] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: stopping tablet replica
I20260925 05:15:09.670646 15597 raft_consensus.cc:2229] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:09.670655 15595 raft_consensus.cc:2229] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:09.670686 15595 raft_consensus.cc:2258] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.670686 15597 raft_consensus.cc:2258] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.670760 15486 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a2d12da19647eabeb7b93ae8968fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:09 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55394
I20260925 05:15:09.670837 15596 tablet_replica.cc:330] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: stopping tablet replica
I20260925 05:15:09.670878 15596 raft_consensus.cc:2229] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:09.670979 15596 raft_consensus.cc:2258] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:09.671054 15595 ts_tablet_manager.cc:1905] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.671221 15597 ts_tablet_manager.cc:1905] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.672619 15596 ts_tablet_manager.cc:1905] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:09.672811 15595 ts_tablet_manager.cc:1918] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:09.672873 15595 log.cc:1196] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/e4a2d12da19647eabeb7b93ae8968fda
I20260925 05:15:09.673094 15595 ts_tablet_manager.cc:1939] T e4a2d12da19647eabeb7b93ae8968fda P d74e9af1e5cc450e96efa170e3dd557d: Deleting consensus metadata
I20260925 05:15:09.673429 15053 catalog_manager.cc:4841] TS d74e9af1e5cc450e96efa170e3dd557d (127.13.160.129:37429): tablet e4a2d12da19647eabeb7b93ae8968fda (table test-table [id=f8307b2996764eb08c1cb9886918bca6]) successfully deleted
I20260925 05:15:09.674121 15597 ts_tablet_manager.cc:1918] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:09.674183 15597 log.cc:1196] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/e4a2d12da19647eabeb7b93ae8968fda
I20260925 05:15:09.674350 15597 ts_tablet_manager.cc:1939] T e4a2d12da19647eabeb7b93ae8968fda P ee0239652cb74311b8461fd1905cea8f: Deleting consensus metadata
I20260925 05:15:09.674651 15596 ts_tablet_manager.cc:1918] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:09.674714 15596 log.cc:1196] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/e4a2d12da19647eabeb7b93ae8968fda
I20260925 05:15:09.674698 15053 catalog_manager.cc:4841] TS ee0239652cb74311b8461fd1905cea8f (127.13.160.130:39555): tablet e4a2d12da19647eabeb7b93ae8968fda (table test-table [id=f8307b2996764eb08c1cb9886918bca6]) successfully deleted
I20260925 05:15:09.674950 15596 ts_tablet_manager.cc:1939] T e4a2d12da19647eabeb7b93ae8968fda P 45624b02321b4436b449f062e38bb715: Deleting consensus metadata
I20260925 05:15:09.675287 15053 catalog_manager.cc:4841] TS 45624b02321b4436b449f062e38bb715 (127.13.160.131:37941): tablet e4a2d12da19647eabeb7b93ae8968fda (table test-table [id=f8307b2996764eb08c1cb9886918bca6]) successfully deleted
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:09.680853 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.671256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.13.160.129:37429)
0925 05:15:09.672322 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.672324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.672331 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:15:09.672346 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.672347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.672374 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.672455 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.673136 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.673705 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.674662 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.674965 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.675026 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.677901 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.677912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.677913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.677921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.679643 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.680021 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.680026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.680026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.680095 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.680294 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.680295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.680295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.680442 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.680587 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.680665 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.680746 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047}
W20260925 05:15:09.681133 15259 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33010
I20260925 05:15:09.688393 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:09.682131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.13.160.190:44065)
0925 05:15:09.682181 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:09.682183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:09.682192 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:09.682265 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:09.682266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:09.682276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:09.682331 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:09.682834 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.683654 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.684493 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:09.684703 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:09.684777 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:09.685942 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:09.685944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:09.685944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:09.685952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:09.687298 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.687638 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.687639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.687640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.687697 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:09.687889 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:09.687890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:09.687890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:09.688003 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:09.688115 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:09.688218 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 05:15:09.688295 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 05:15:09.688639 15068 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47230
I20260925 05:15:09.689304 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15141
I20260925 05:15:09.695350 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15281
I20260925 05:15:09.702272 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15421
I20260925 05:15:09.707618 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15037
2026-09-25T05:15:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2303 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15631](info): set up 2 sockets
Sep 25 05:15:09 dist-test-slave-9tgz krb5kdc[15631](info): commencing operation
krb5kdc: starting...
W20260925 05:15:10.743422 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:10 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:10.866262 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40279
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:10.949708 15644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:10.949991 15644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:10.950033 15644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:10.951385 15644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:10.951455 15644 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 05:15:10.951479 15644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:10.951534 15644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:10.951566 15644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:10.951589 15644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:10.953330 15644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40279
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:10.953652 15644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:10.953904 15644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:10.959050 15649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:10.959050 15652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:10.959132 15650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:10.959414 15644 server_base.cc:1034] running on GCE node
I20260925 05:15:10.959645 15644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:10.959952 15644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:10.961104 15644 hybrid_clock.cc:648] HybridClock initialized: now 1790313310961090 us; error 31 us; skew 500 ppm
Sep 25 05:15:10 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313310, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:10.963719 15644 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:10.964192 15644 webserver.cc:460] Webserver started at http://127.13.160.190:46375/ using document root <none> and password file <none>
I20260925 05:15:10.964432 15644 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:10.964514 15644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:10.964670 15644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:10.965811 15644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "613fc87160a64a89b5d7811bc9cca52e"
format_stamp: "Formatted at 2026-09-25 05:15:10 on dist-test-slave-9tgz"
server_key: "f0e0df76f7e0aa51f0e2ff0bd3729469"
server_key_iv: "307582712bf5a288388ded05b4392247"
server_key_version: "encryptionkey@0"
I20260925 05:15:10.966207 15644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "613fc87160a64a89b5d7811bc9cca52e"
format_stamp: "Formatted at 2026-09-25 05:15:10 on dist-test-slave-9tgz"
server_key: "f0e0df76f7e0aa51f0e2ff0bd3729469"
server_key_iv: "307582712bf5a288388ded05b4392247"
server_key_version: "encryptionkey@0"
I20260925 05:15:10.967684 15644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:10.968451 15659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:10.968628 15644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:10.968741 15644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "613fc87160a64a89b5d7811bc9cca52e"
format_stamp: "Formatted at 2026-09-25 05:15:10 on dist-test-slave-9tgz"
server_key: "f0e0df76f7e0aa51f0e2ff0bd3729469"
server_key_iv: "307582712bf5a288388ded05b4392247"
server_key_version: "encryptionkey@0"
I20260925 05:15:10.968823 15644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:10.979101 15644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:10.981220 15644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:10.981417 15644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:10.986313 15644 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40279
I20260925 05:15:10.986363 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40279 every 8 connection(s)
I20260925 05:15:10.986739 15644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:10.987511 15722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:10.989909 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Bootstrap starting.
I20260925 05:15:10.990554 15722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:10.990875 15722 log.cc:824] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:10.991472 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15644
I20260925 05:15:10.991571 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:10.991739 13954 external_mini_cluster.cc:1428] Setting key dacaf55cddca807bdac8d521f958be43
I20260925 05:15:10.991815 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: No bootstrap required, opened a new log
I20260925 05:15:10.994779 15722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } }
I20260925 05:15:10.994998 15722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:10.995038 15722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 613fc87160a64a89b5d7811bc9cca52e, State: Initialized, Role: FOLLOWER
I20260925 05:15:10.995178 15722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } }
I20260925 05:15:10.995249 15722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:10.995276 15722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:10.995321 15722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:10.996033 15722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } }
I20260925 05:15:10.996129 15722 leader_election.cc:302] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 613fc87160a64a89b5d7811bc9cca52e; no voters:
I20260925 05:15:10.996332 15722 leader_election.cc:288] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:10.996390 15727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:10.996548 15727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [term 1 LEADER]: Becoming Leader. State: Replica: 613fc87160a64a89b5d7811bc9cca52e, State: Running, Role: LEADER
I20260925 05:15:10.996631 15722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:10.996642 15727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } }
I20260925 05:15:10.997007 15728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "613fc87160a64a89b5d7811bc9cca52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } } }
I20260925 05:15:10.997097 15728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:10.996997 15727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 613fc87160a64a89b5d7811bc9cca52e. Latest consensus state: current_term: 1 leader_uuid: "613fc87160a64a89b5d7811bc9cca52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613fc87160a64a89b5d7811bc9cca52e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40279 } } }
I20260925 05:15:10.997289 15727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:10.997417 15733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 05:15:10 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:10.999680 15733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:11.001539 15733 catalog_manager.cc:1347] Generated new cluster ID: c46d5cfaf58c46bc83ff72545a29b209
I20260925 05:15:11.003171 15733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:11.004099 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:10.992305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.13.160.190:40279)
0925 05:15:10.992501 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:10.992513 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:10.992555 (+ 42us) server_negotiation.cc:407] Connection header received
0925 05:15:10.993285 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:10.993295 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:10.993366 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:10.993539 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:10.994141 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:10.995547 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:10.996481 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:10.997135 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:10.997251 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:10.999978 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:10.999983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:10.999983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:10.999996 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.002386 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.002767 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.002770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.002771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.002885 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.003205 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.003206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.003207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.003488 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.003692 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.003776 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.003920 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260925 05:15:11.037012 15733 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:11.037842 15733 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:11.045322 15733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Generated new TSK 0
I20260925 05:15:11.045562 15733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:11.077502 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40279
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:11.195849 15748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:11.196170 15748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:11.196228 15748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:11.197582 15748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:11.197638 15748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:11.197662 15748 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 05:15:11.197716 15748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:11.197762 15748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:11.199544 15748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:40279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:11.199857 15748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:11.200138 15748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:11.205888 15756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:11.205895 15753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:11.205895 15754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:11.206138 15748 server_base.cc:1034] running on GCE node
I20260925 05:15:11.206318 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:11.206547 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:11.207717 15748 hybrid_clock.cc:648] HybridClock initialized: now 1790313311207706 us; error 33 us; skew 500 ppm
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:11.210315 15748 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:11.210850 15748 webserver.cc:460] Webserver started at http://127.13.160.129:40663/ using document root <none> and password file <none>
I20260925 05:15:11.211135 15748 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:11.211208 15748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:11.211360 15748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:11.212471 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "47c0616f8798765db1c3d6b9b3118db4"
server_key_iv: "ae582760ae0264e5c1565b28d041c6a4"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.212845 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "47c0616f8798765db1c3d6b9b3118db4"
server_key_iv: "ae582760ae0264e5c1565b28d041c6a4"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.214254 15748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:11.215063 15763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.215245 15748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:11.215348 15748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "47c0616f8798765db1c3d6b9b3118db4"
server_key_iv: "ae582760ae0264e5c1565b28d041c6a4"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.215433 15748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:11.248760 15748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:11.250718 15748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:11.250917 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:11.251451 15748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:11.251538 15748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.251585 15748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:11.251631 15748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:11.259006 15748 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40195
I20260925 05:15:11.259099 15878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40195 every 8 connection(s)
I20260925 05:15:11.259413 15748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:11.261875 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.252029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:46771 (local address 127.13.160.190:40279)
0925 05:15:11.252099 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.252102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.252326 (+ 224us) server_negotiation.cc:407] Connection header received
0925 05:15:11.252942 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.252943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.252958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.253046 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.254029 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.254954 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.256050 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.256408 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.256492 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.259071 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.259074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.259074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.259083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.260699 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.261092 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.261093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.261093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.261146 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.261362 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.261362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.261363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.261466 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.261530 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.261732 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.261806 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:15:11.262045 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.252014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.129:46771)
0925 05:15:11.252190 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.252230 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.252786 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.253146 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.253150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.253573 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.253912 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.253919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.255058 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.255059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.255975 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.255978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.256511 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.256512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.256665 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.257302 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.257312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.258947 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.260778 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.260784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.260786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.261022 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.261213 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.261214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.261214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.261310 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.261519 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.261522 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.261643 (+ 121us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.261721 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.261844 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260925 05:15:11.262542 15879 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40279
I20260925 05:15:11.262650 15879 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:11.262836 15879 heartbeater.cc:511] Master 127.13.160.190:40279 requested a full tablet report, sending...
I20260925 05:15:11.263422 15676 ts_manager.cc:194] Registered new tserver with Master: cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195)
I20260925 05:15:11.263963 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15748
I20260925 05:15:11.264070 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:11.264211 15676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:46771
I20260925 05:15:11.264246 13954 external_mini_cluster.cc:1428] Setting key 6dea4b45adb25c779be9fc93993ba79e
I20260925 05:15:11.270828 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.263763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:44837 (local address 127.13.160.190:40279)
0925 05:15:11.263807 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.263810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.264620 (+ 810us) server_negotiation.cc:407] Connection header received
0925 05:15:11.264702 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.264703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.264714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.264775 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.265089 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.265632 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.266720 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.267058 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.267125 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.268198 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.268199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.268199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.268205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.269676 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.270105 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.270106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.270106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.270155 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.270364 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.270364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.270365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.270464 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.270510 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.270700 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.270762 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:11.270968 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.263694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.129:44837)
0925 05:15:11.264542 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.264570 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.264647 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.264777 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.264778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.264961 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.265014 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.265016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.265730 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.265731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.266630 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.266633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.267132 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.267133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.267189 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.267677 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.267684 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.268114 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.269763 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.269764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.269765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.270022 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.270217 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.270218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.270218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.270295 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.270532 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.270533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.270594 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.270684 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.270810 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:11.290751 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40279
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:11.376698 15888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:11.376947 15888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:11.376992 15888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:11.378626 15888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:11.378700 15888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:11.378737 15888 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 05:15:11.378765 15888 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:11.378795 15888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:11.381040 15888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:40279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:11.381369 15888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:11.381676 15888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:11.387347 15888 server_base.cc:1034] running on GCE node
W20260925 05:15:11.387329 15894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:11.387384 15896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:11.387331 15893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:11.387765 15888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:11.388044 15888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:11.389199 15888 hybrid_clock.cc:648] HybridClock initialized: now 1790313311389182 us; error 43 us; skew 500 ppm
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:11.391779 15888 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:11.392256 15888 webserver.cc:460] Webserver started at http://127.13.160.130:38421/ using document root <none> and password file <none>
I20260925 05:15:11.392505 15888 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:11.392577 15888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:11.392731 15888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:11.393837 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "3b13135cca2449f990db3b1a9a8c9f91"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "eb4e009028487fbe9af30cc1af008a33"
server_key_iv: "70f3d142a7ae26799364d0f99d7d7f95"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.394215 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "3b13135cca2449f990db3b1a9a8c9f91"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "eb4e009028487fbe9af30cc1af008a33"
server_key_iv: "70f3d142a7ae26799364d0f99d7d7f95"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.395776 15888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:11.396692 15903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.396869 15888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:11.396975 15888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "3b13135cca2449f990db3b1a9a8c9f91"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "eb4e009028487fbe9af30cc1af008a33"
server_key_iv: "70f3d142a7ae26799364d0f99d7d7f95"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.397070 15888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:11.405046 15888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:11.407169 15888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:11.407346 15888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:11.407886 15888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:11.407950 15888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.407989 15888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:11.408031 15888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.415364 15888 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38221
I20260925 05:15:11.415445 16018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38221 every 8 connection(s)
I20260925 05:15:11.415772 15888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:11.415961 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 15888
I20260925 05:15:11.416059 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:11.416244 13954 external_mini_cluster.cc:1428] Setting key c1642aba02625594b0d926eb852aa019
I20260925 05:15:11.420099 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.408462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:59545 (local address 127.13.160.190:40279)
0925 05:15:11.408515 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.408517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.408764 (+ 247us) server_negotiation.cc:407] Connection header received
0925 05:15:11.409271 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.409273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.409290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.409369 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.410318 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.411221 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.412314 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.412651 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.412731 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.416554 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.416556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.416557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.416565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.418401 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.418939 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.418941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.418941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.419025 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.419362 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.419363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.419363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.419543 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.419630 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.419942 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.420030 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:15:11.420392 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.408479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.130:59545)
0925 05:15:11.408645 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.408682 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.409166 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.409484 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.409488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.409879 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.410204 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.410210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.411342 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.411343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.412208 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.412211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.412775 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.412777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.412967 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.414367 (+ 1400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.414380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.416406 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.418551 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.418562 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.418565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.418828 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.419102 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.419103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.419119 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.419263 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.419623 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.419628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.419782 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.419904 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.420096 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":104,"threads_started":1}
I20260925 05:15:11.420940 16019 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40279
I20260925 05:15:11.421038 16019 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:11.421243 16019 heartbeater.cc:511] Master 127.13.160.190:40279 requested a full tablet report, sending...
I20260925 05:15:11.421666 15675 ts_manager.cc:194] Registered new tserver with Master: 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221)
I20260925 05:15:11.422178 15675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:59545
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
I20260925 05:15:11.429091 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.422561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:48857 (local address 127.13.160.190:40279)
0925 05:15:11.422612 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.422614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.423054 (+ 440us) server_negotiation.cc:407] Connection header received
0925 05:15:11.423120 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.423121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.423132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.423192 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.423611 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.424089 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.425160 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.425350 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.425398 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.426477 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.426479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.426479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.426488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.427849 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.428222 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.428224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.428225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.428283 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.428484 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.428486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.428486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.428643 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.428729 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.428907 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.428993 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:11.429105 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.422794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.130:48857)
0925 05:15:11.422966 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.422979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.423051 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.423212 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.423213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.423454 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.423536 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.423538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.424192 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.424193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.425076 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.425079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.425428 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.425429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.425522 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.425989 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.425995 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.426399 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.427949 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.427950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.427950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.428133 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.428352 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.428353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.428354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.428419 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.428764 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.428764 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.428803 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.428882 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.428991 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:11.444073 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40279
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:11.527870 16028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:11.528112 16028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:11.528139 16028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:11.529443 16028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:11.529490 16028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:11.529512 16028 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 05:15:11.529568 16028 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:11.529595 16028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:11.531263 16028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:40279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:11.531550 16028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:11.531797 16028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:11.537014 16033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:11.537046 16028 server_base.cc:1034] running on GCE node
W20260925 05:15:11.537029 16036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:11.537020 16034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:11.537391 16028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:11.537679 16028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:11.538838 16028 hybrid_clock.cc:648] HybridClock initialized: now 1790313311538827 us; error 49 us; skew 500 ppm
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:11.541334 16028 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:11.541857 16028 webserver.cc:460] Webserver started at http://127.13.160.131:33347/ using document root <none> and password file <none>
I20260925 05:15:11.542090 16028 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:11.542161 16028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:11.542315 16028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:11.543467 16028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "f460f058ed30439eb57d5931debda31e"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "ca38f26206331f992ac80c100362db52"
server_key_iv: "2367da39f0823ca64645624bd41e3e9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.543839 16028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "f460f058ed30439eb57d5931debda31e"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "ca38f26206331f992ac80c100362db52"
server_key_iv: "2367da39f0823ca64645624bd41e3e9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.545310 16028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:11.546132 16043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.546307 16028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:11.546417 16028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "f460f058ed30439eb57d5931debda31e"
format_stamp: "Formatted at 2026-09-25 05:15:11 on dist-test-slave-9tgz"
server_key: "ca38f26206331f992ac80c100362db52"
server_key_iv: "2367da39f0823ca64645624bd41e3e9b"
server_key_version: "encryptionkey@0"
I20260925 05:15:11.546497 16028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:11.555719 16028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:11.557873 16028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:11.558045 16028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:11.558701 16028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:11.558753 16028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.558796 16028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:11.558811 16028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.565920 16028 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:37587
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:11.565991 16158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:37587 every 8 connection(s)
I20260925 05:15:11.566365 16028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:11.569190 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16028
I20260925 05:15:11.569281 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:11.569468 13954 external_mini_cluster.cc:1428] Setting key e012d8482c1935b300e2263a2948f178
I20260925 05:15:11.569664 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.559195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:33539 (local address 127.13.160.190:40279)
0925 05:15:11.559266 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.559269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.559486 (+ 217us) server_negotiation.cc:407] Connection header received
0925 05:15:11.560048 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.560050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.560064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.560115 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.560813 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.561640 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.563284 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.563473 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.563571 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.566589 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.566591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.566591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.566600 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.568241 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.568637 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.568638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.568638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.568690 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.568908 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.568908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.568909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.569035 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.569086 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.569449 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.569561 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:15:11.569872 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.559186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.131:33539)
0925 05:15:11.559367 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.559409 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.559922 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.560195 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.560199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.560496 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.560714 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.560718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.561747 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.561748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.563151 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.563154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.563594 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.563595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.563754 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.564376 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.564392 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.566447 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.568325 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.568330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.568333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.568565 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.568751 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.568752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.568752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.568854 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.569100 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.569106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.569291 (+ 185us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.569422 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.569606 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260925 05:15:11.570393 16159 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40279
I20260925 05:15:11.570511 16159 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:11.570703 16159 heartbeater.cc:511] Master 127.13.160.190:40279 requested a full tablet report, sending...
I20260925 05:15:11.571094 15676 ts_manager.cc:194] Registered new tserver with Master: f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:11.571583 15676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:33539
I20260925 05:15:11.577564 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.571515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.13.160.131:36553)
0925 05:15:11.571698 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.571709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.571768 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.571916 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.571917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.572131 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.572211 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.572213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.572918 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.572919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.573442 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.573444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.573839 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.573840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.573920 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.574427 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.574432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.574739 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.576474 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.576476 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.576476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.576655 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.576880 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.576881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.576881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.576942 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.577207 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.577208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.577259 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.577346 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.577457 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260925 05:15:11.577564 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.571576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36553 (local address 127.13.160.190:40279)
0925 05:15:11.571623 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.571625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.571761 (+ 136us) server_negotiation.cc:407] Connection header received
0925 05:15:11.571823 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.571823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.571831 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.571901 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.572287 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.572827 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.573550 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.573757 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.573830 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.574850 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.574852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.574852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.574860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.576366 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.576744 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.576746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.576747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.576805 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.577013 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.577015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.577015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.577139 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.577216 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.577371 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.577455 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:11.581159 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:11.604686 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.597615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46632 (local address 127.13.160.190:40279)
0925 05:15:11.597682 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.597684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.597693 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:11.597718 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.597719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.597728 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.597780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.598422 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.599002 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.599733 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.599941 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.600020 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.602401 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.602403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.602403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.602410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.603743 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.604058 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.604059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.604059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.604108 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.604268 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.604269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.604269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.604372 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.604447 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.604557 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.604616 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:15:11.606383 15676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:11.608144 15676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:11.615196 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.13.160.130:38221)
0925 05:15:11.611172 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.611178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.611188 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:11.611236 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.611238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.611255 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.611712 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:11.611780 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.612484 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.613885 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.614646 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614749 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.614806 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.615022 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.615076 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":82,"threads_started":1}
I20260925 05:15:11.615180 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38221 (local address 127.0.0.1:37284)
0925 05:15:11.610745 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.610778 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.610868 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.611544 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.611545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.611553 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:11.611611 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.611615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.612576 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.612578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.613795 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.613798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.614747 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.614830 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.614851 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.615002 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.615068 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260925 05:15:11.615329 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.0.0.1:55690)
0925 05:15:11.611423 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.611433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.611487 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.611698 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.611699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.611700 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:11.611736 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.611737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.613025 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.613026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.614053 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.614056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.615068 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.615069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.615128 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.615132 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.615184 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.615235 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260925 05:15:11.615411 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37587 (local address 127.0.0.1:37756)
0925 05:15:11.610990 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.611001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.611058 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.611356 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.611359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.611361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:11.611436 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.611438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.613015 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.613016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.614057 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.614059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.615025 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.615026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.615081 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.615084 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.615143 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.615175 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":22,"thread_start_us":78,"threads_started":1}
I20260925 05:15:11.615600 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.13.160.129:40195)
0925 05:15:11.611182 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.611187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.611532 (+ 345us) server_negotiation.cc:407] Connection header received
0925 05:15:11.611584 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.611587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.611602 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.611677 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:11.611820 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.612920 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614154 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.614864 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614981 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.615032 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.615451 (+ 419us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.615493 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260925 05:15:11.615726 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.610975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.13.160.131:37587)
0925 05:15:11.611203 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.611207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.611215 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:11.611249 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.611251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.611265 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.611338 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:11.611738 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.612916 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614143 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.614651 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.614745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.614806 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.615225 (+ 419us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.615612 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260925 05:15:11.616421 15813 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (DEFAULT_TABLE table=test-table [id=842527aa2aba49888e71a378539b2b66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:11.616433 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (DEFAULT_TABLE table=test-table [id=842527aa2aba49888e71a378539b2b66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:11.616596 15953 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (DEFAULT_TABLE table=test-table [id=842527aa2aba49888e71a378539b2b66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:11.616739 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9734c77e2c4ca9b71fcf5eda783f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.616793 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9734c77e2c4ca9b71fcf5eda783f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.616863 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9734c77e2c4ca9b71fcf5eda783f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.620379 16182 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Bootstrap starting.
I20260925 05:15:11.620548 16184 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Bootstrap starting.
I20260925 05:15:11.620752 16183 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Bootstrap starting.
I20260925 05:15:11.621204 16182 tablet_bootstrap.cc:654] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:11.621227 16184 tablet_bootstrap.cc:654] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:11.621524 16182 log.cc:824] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:11.621615 16184 log.cc:824] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:11.621719 16183 tablet_bootstrap.cc:654] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:11.622149 16183 log.cc:824] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:11.622341 16182 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: No bootstrap required, opened a new log
I20260925 05:15:11.622383 16184 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: No bootstrap required, opened a new log
I20260925 05:15:11.622440 16182 ts_tablet_manager.cc:1397] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:11.622442 16184 ts_tablet_manager.cc:1397] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:11.622831 16183 tablet_bootstrap.cc:492] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: No bootstrap required, opened a new log
I20260925 05:15:11.622926 16183 ts_tablet_manager.cc:1397] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:11.623960 16182 raft_consensus.cc:348] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.624119 16182 raft_consensus.cc:374] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.624167 16182 raft_consensus.cc:729] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.624306 16182 consensus_queue.cc:260] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.624578 16182 ts_tablet_manager.cc:1428] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:11.624598 15879 heartbeater.cc:503] Master 127.13.160.190:40279 was elected leader, sending a full tablet report...
I20260925 05:15:11.624519 16184 raft_consensus.cc:348] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.624681 16184 raft_consensus.cc:374] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.624715 16184 raft_consensus.cc:729] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.624830 16184 consensus_queue.cc:260] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.625061 16184 ts_tablet_manager.cc:1428] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:11.625062 16183 raft_consensus.cc:348] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.625207 16019 heartbeater.cc:503] Master 127.13.160.190:40279 was elected leader, sending a full tablet report...
I20260925 05:15:11.625236 16183 raft_consensus.cc:374] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.625277 16183 raft_consensus.cc:729] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f460f058ed30439eb57d5931debda31e, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.625406 16183 consensus_queue.cc:260] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.625650 16183 ts_tablet_manager.cc:1428] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:11.625782 16159 heartbeater.cc:503] Master 127.13.160.190:40279 was elected leader, sending a full tablet report...
W20260925 05:15:11.666563 16020 tablet.cc:2369] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:11.696743 16188 raft_consensus.cc:482] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:11.696911 16188 raft_consensus.cc:504] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.697748 16188 leader_election.cc:288] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f460f058ed30439eb57d5931debda31e (127.13.160.131:37587), 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221)
I20260925 05:15:11.701556 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.697677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:52559 (local address 127.13.160.131:37587)
0925 05:15:11.697737 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.697739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.697842 (+ 103us) server_negotiation.cc:407] Connection header received
0925 05:15:11.697963 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.697964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.697974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.698056 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:11.698197 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.698944 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.700531 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.701173 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.701246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.701254 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.701430 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.701477 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:11.701556 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.697677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:55783 (local address 127.13.160.130:38221)
0925 05:15:11.697728 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.697731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.697744 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:15:11.697780 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.697786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.697797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.697904 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:11.698076 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.698936 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.700532 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.701129 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.701215 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.701227 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.701453 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.701493 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:15:11.701556 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.697564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38221 (local address 127.13.160.129:55783)
0925 05:15:11.697622 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.697636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.697687 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.697917 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.697918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.697920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:11.697993 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.697995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.699048 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.699049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.700424 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.700427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.701250 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.701251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.701350 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.701353 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.701403 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.701455 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":121}
I20260925 05:15:11.701566 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.697564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37587 (local address 127.13.160.129:52559)
0925 05:15:11.697786 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.697795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.697869 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.698070 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.698070 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.698072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:11.698129 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.698130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.699052 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.699052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.700424 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.700427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.701250 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.701251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.701334 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.701339 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.701401 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.701455 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":102,"thread_start_us":106,"threads_started":1}
I20260925 05:15:11.702034 15973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9734c77e2c4ca9b71fcf5eda783f26" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b13135cca2449f990db3b1a9a8c9f91" is_pre_election: true
I20260925 05:15:11.702221 15973 raft_consensus.cc:2454] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 0.
I20260925 05:15:11.702461 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9734c77e2c4ca9b71fcf5eda783f26" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e" is_pre_election: true
I20260925 05:15:11.702497 15766 leader_election.cc:302] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, cdb99d2b1e7b4d5a87a38fb344c81118; no voters:
I20260925 05:15:11.702606 16113 raft_consensus.cc:2454] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 0.
I20260925 05:15:11.702716 16188 raft_consensus.cc:2781] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:11.702785 16188 raft_consensus.cc:482] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:11.702809 16188 raft_consensus.cc:3037] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:11.703682 16188 raft_consensus.cc:504] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.703855 16188 leader_election.cc:288] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 election: Requested vote from peers f460f058ed30439eb57d5931debda31e (127.13.160.131:37587), 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221)
I20260925 05:15:11.704005 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9734c77e2c4ca9b71fcf5eda783f26" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e"
I20260925 05:15:11.704093 16113 raft_consensus.cc:3037] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:11.704875 16113 raft_consensus.cc:2454] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 1.
I20260925 05:15:11.704995 15973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9734c77e2c4ca9b71fcf5eda783f26" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b13135cca2449f990db3b1a9a8c9f91"
I20260925 05:15:11.705078 15973 raft_consensus.cc:3037] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:11.705387 15767 leader_election.cc:302] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdb99d2b1e7b4d5a87a38fb344c81118, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:11.705550 16188 raft_consensus.cc:2781] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:11.705751 16188 raft_consensus.cc:686] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 LEADER]: Becoming Leader. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Running, Role: LEADER
I20260925 05:15:11.705864 16188 consensus_queue.cc:237] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } }
I20260925 05:15:11.706027 15973 raft_consensus.cc:2454] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 1.
I20260925 05:15:11.706768 15676 catalog_manager.cc:5495] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } health_report { overall_health: HEALTHY } } }
Sep 25 05:15:11 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313310, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:11.754326 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.747690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.13.160.190:40279)
0925 05:15:11.747790 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:11.747791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:11.747828 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:15:11.747888 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:11.747888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:11.747897 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:11.747948 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:11.748413 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.748893 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.749565 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.749760 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.749839 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.751951 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:11.751952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:11.751952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:11.751959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:11.753358 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.753712 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.753714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.753714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.753762 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:11.753924 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:11.753924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:11.753924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:11.754024 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:11.754072 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:11.754191 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 05:15:11.754258 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 05:15:11.754489 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:11.747609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40279 (local address 127.0.0.1:46638)
0925 05:15:11.747767 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:11.747777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:11.747843 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:11.747968 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:11.747969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:11.748277 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:11.748328 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.748329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.748963 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.748964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.749492 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:11.749494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:11.749838 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:11.749839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:11.749891 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:11.750395 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:11.750399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:11.751868 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:11.753435 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.753441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.753443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.753648 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.753818 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:11.753818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:11.753818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:11.753873 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:11.754076 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:11.754081 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:11.754124 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 05:15:11.754175 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:11.754326 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":59,"thread_start_us":75,"threads_started":1}
I20260925 05:15:11.756423 15676 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:46638:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 05:15:11.756671 15676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:11.758256 15813 tablet_service.cc:1467] Processing CreateTablet for tablet f606dc081c7c46d288524708b8cb21a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=617519ea7b754f27a809c1cb2a79ca7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:11.758257 15953 tablet_service.cc:1467] Processing CreateTablet for tablet f606dc081c7c46d288524708b8cb21a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=617519ea7b754f27a809c1cb2a79ca7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:11.758253 16093 tablet_service.cc:1467] Processing CreateTablet for tablet f606dc081c7c46d288524708b8cb21a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=617519ea7b754f27a809c1cb2a79ca7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:11.758440 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f606dc081c7c46d288524708b8cb21a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.758440 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f606dc081c7c46d288524708b8cb21a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.758440 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f606dc081c7c46d288524708b8cb21a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:11.759684 16182 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: Bootstrap starting.
I20260925 05:15:11.759981 16183 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: Bootstrap starting.
I20260925 05:15:11.759982 16184 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: Bootstrap starting.
I20260925 05:15:11.760144 16182 tablet_bootstrap.cc:654] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:11.760561 16183 tablet_bootstrap.cc:654] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:11.760627 16184 tablet_bootstrap.cc:654] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: Neither blocks nor log segments found. Creating new log.
W20260925 05:15:11.760999 15880 tablet.cc:2369] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:11.761077 16182 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: No bootstrap required, opened a new log
I20260925 05:15:11.761133 16182 ts_tablet_manager.cc:1397] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:11.761354 16184 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: No bootstrap required, opened a new log
I20260925 05:15:11.761296 16182 raft_consensus.cc:348] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.761396 16182 raft_consensus.cc:374] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.761409 16184 ts_tablet_manager.cc:1397] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:11.761421 16182 raft_consensus.cc:729] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.761477 16182 consensus_queue.cc:260] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.761560 16184 raft_consensus.cc:348] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.761602 16182 ts_tablet_manager.cc:1428] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.761634 16184 raft_consensus.cc:374] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.761662 16184 raft_consensus.cc:729] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.761654 16183 tablet_bootstrap.cc:492] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: No bootstrap required, opened a new log
I20260925 05:15:11.761703 16183 ts_tablet_manager.cc:1397] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:11.761703 16184 consensus_queue.cc:260] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.761797 16184 ts_tablet_manager.cc:1428] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:11.761862 16183 raft_consensus.cc:348] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.761885 16189 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:11.761941 16183 raft_consensus.cc:374] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:11.761967 16183 raft_consensus.cc:729] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f460f058ed30439eb57d5931debda31e, State: Initialized, Role: FOLLOWER
I20260925 05:15:11.761960 16189 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:11.762020 16183 consensus_queue.cc:260] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:11.762109 16183 ts_tablet_manager.cc:1428] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:11.762209 16190 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:11.762235 16188 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:11.762305 16190 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:11.762320 16188 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 05:15:11.817028 16160 tablet.cc:2369] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:12.099466 16188 consensus_queue.cc:1035] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.101824 16188 consensus_queue.cc:1035] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.114398 16188 raft_consensus.cc:482] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:12.114491 16188 raft_consensus.cc:504] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.114666 16188 leader_election.cc:288] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:12.114848 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f606dc081c7c46d288524708b8cb21a7" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e" is_pre_election: true
I20260925 05:15:12.114984 16113 raft_consensus.cc:2454] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 0.
I20260925 05:15:12.114868 15973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f606dc081c7c46d288524708b8cb21a7" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b13135cca2449f990db3b1a9a8c9f91" is_pre_election: true
I20260925 05:15:12.115052 15973 raft_consensus.cc:2454] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 0.
I20260925 05:15:12.115139 15767 leader_election.cc:302] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdb99d2b1e7b4d5a87a38fb344c81118, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:12.115271 16188 raft_consensus.cc:2781] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:12.115329 16188 raft_consensus.cc:482] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:12.115377 16188 raft_consensus.cc:3037] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.116129 16188 raft_consensus.cc:504] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.116269 16188 leader_election.cc:288] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 election: Requested vote from peers 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:12.116413 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f606dc081c7c46d288524708b8cb21a7" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e"
I20260925 05:15:12.116478 16113 raft_consensus.cc:3037] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.117026 16113 raft_consensus.cc:2454] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 1.
I20260925 05:15:12.117144 15973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f606dc081c7c46d288524708b8cb21a7" candidate_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b13135cca2449f990db3b1a9a8c9f91"
I20260925 05:15:12.117173 15767 leader_election.cc:302] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdb99d2b1e7b4d5a87a38fb344c81118, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:12.117223 15973 raft_consensus.cc:3037] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.117321 16188 raft_consensus.cc:2781] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:12.117398 16188 raft_consensus.cc:686] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 LEADER]: Becoming Leader. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Running, Role: LEADER
I20260925 05:15:12.117447 16188 consensus_queue.cc:237] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.117667 16192 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: TxnStatusTablet state changed. Reason: New leader cdb99d2b1e7b4d5a87a38fb344c81118. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.117760 16192 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:12.117938 15675 catalog_manager.cc:5495] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:12.118058 15973 raft_consensus.cc:2454] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb99d2b1e7b4d5a87a38fb344c81118 in term 1.
I20260925 05:15:12.118311 16209 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 05:15:12.118454 16209 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 05:15:12.206120 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.203217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55694 (local address 127.13.160.129:40195)
0925 05:15:12.203341 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.203343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.203351 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:12.203381 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.203382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.203401 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.203468 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.203676 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.204195 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.205311 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.205527 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.205585 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.205790 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.205888 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.205948 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.206029 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.206050 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260925 05:15:12.206158 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.202944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.0.0.1:55694)
0925 05:15:12.203039 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.203053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.203109 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.203508 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.203509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.203511 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:12.203569 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.203572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.204291 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.204293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.205208 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.205212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.205606 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.205607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.205675 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.205695 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.205953 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.205961 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.206022 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.206077 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 05:15:12.207790 15973 raft_consensus.cc:1261] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Refusing update from remote peer cdb99d2b1e7b4d5a87a38fb344c81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:12.207790 16113 raft_consensus.cc:1261] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Refusing update from remote peer cdb99d2b1e7b4d5a87a38fb344c81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:12.208029 16188 consensus_queue.cc:1035] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.208127 16192 consensus_queue.cc:1035] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.209151 16190 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: TxnStatusTablet state changed. Reason: New leader cdb99d2b1e7b4d5a87a38fb344c81118. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.209252 16190 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:12.209200 16189 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: TxnStatusTablet state changed. Reason: New leader cdb99d2b1e7b4d5a87a38fb344c81118. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.209301 16189 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:12.210259 16192 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.210361 16192 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:12.210474 16189 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.210554 16189 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P 3b13135cca2449f990db3b1a9a8c9f91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:12.210265 16188 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.210678 16188 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P cdb99d2b1e7b4d5a87a38fb344c81118: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:12.210819 16205 mvcc.cc:205] Tried to move back new op lower bound from 7333123326800908288 to 7333123326433251328. Current Snapshot: MvccSnapshot[applied={T|T < 7333123326800908288}]
I20260925 05:15:12.211436 16204 mvcc.cc:205] Tried to move back new op lower bound from 7333123326800908288 to 7333123326433251328. Current Snapshot: MvccSnapshot[applied={T|T < 7333123326800908288 or (T in {7333123326800908288})}]
I20260925 05:15:12.211561 16207 mvcc.cc:205] Tried to move back new op lower bound from 7333123326800908288 to 7333123326433251328. Current Snapshot: MvccSnapshot[applied={T|T < 7333123326800908288}]
I20260925 05:15:12.211642 16190 tablet_replica.cc:439] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } } }
I20260925 05:15:12.211714 16190 tablet_replica.cc:442] T f606dc081c7c46d288524708b8cb21a7 P f460f058ed30439eb57d5931debda31e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:12.214779 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.212300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.13.160.129:40195)
0925 05:15:12.212340 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.212342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.212351 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:12.212377 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.212378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.212387 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.212444 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.212649 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.213224 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.213977 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.214305 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.214381 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.214506 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.214543 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.214599 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.214651 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.214679 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:15:12.220465 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.217477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33699 (local address 127.13.160.129:40195)
0925 05:15:12.217542 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.217544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.217738 (+ 194us) server_negotiation.cc:407] Connection header received
0925 05:15:12.217761 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.217762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.217772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.217815 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.217987 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.218623 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.219722 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.219900 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.219959 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.220187 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.220213 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.220253 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.220336 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.220374 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":42}
I20260925 05:15:12.220567 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.217390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.13.160.129:33699)
0925 05:15:12.217617 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.217628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.217696 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.217836 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.217837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.217839 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:12.217922 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.217923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.218710 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.218711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.219634 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.219638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.219978 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.219979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.220061 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.220078 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.220278 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.220279 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.220334 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.220397 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":2,"thread_start_us":128,"threads_started":1}
I20260925 05:15:12.235253 15675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:12.235388 15675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:12.236194 15675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Sending DeleteTablet for 3 replicas of tablet 5e9734c77e2c4ca9b71fcf5eda783f26
I20260925 05:15:12.236532 15953 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:12 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37284
I20260925 05:15:12.236569 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:12 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37756
I20260925 05:15:12.236569 15813 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9734c77e2c4ca9b71fcf5eda783f26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:12 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55690
I20260925 05:15:12.236789 16225 tablet_replica.cc:330] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: stopping tablet replica
I20260925 05:15:12.236903 16225 raft_consensus.cc:2229] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:12.236971 16225 raft_consensus.cc:2258] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:12.237083 16227 tablet_replica.cc:330] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: stopping tablet replica
I20260925 05:15:12.237030 15675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:12.237211 15675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:12.237442 16225 ts_tablet_manager.cc:1905] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:12.237489 16227 raft_consensus.cc:2229] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:12.237717 16227 raft_consensus.cc:2258] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:12.238179 16227 ts_tablet_manager.cc:1905] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:12.238484 15813 tablet_service.cc:1467] Processing CreateTablet for tablet 56e9bc74147047499f28428877f58d34 (DEFAULT_TABLE table=test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:12.238615 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e9bc74147047499f28428877f58d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:12.239488 16225 ts_tablet_manager.cc:1918] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:12.239573 16225 log.cc:1196] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/5e9734c77e2c4ca9b71fcf5eda783f26
I20260925 05:15:12.239830 16225 ts_tablet_manager.cc:1939] T 5e9734c77e2c4ca9b71fcf5eda783f26 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting consensus metadata
I20260925 05:15:12.240028 15953 tablet_service.cc:1467] Processing CreateTablet for tablet 56e9bc74147047499f28428877f58d34 (DEFAULT_TABLE table=test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:12.240146 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e9bc74147047499f28428877f58d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:12.240280 16227 ts_tablet_manager.cc:1918] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:12.240276 15662 catalog_manager.cc:4841] TS 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221): tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (table test-table [id=842527aa2aba49888e71a378539b2b66]) successfully deleted
I20260925 05:15:12.240392 16227 log.cc:1196] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/5e9734c77e2c4ca9b71fcf5eda783f26
I20260925 05:15:12.240656 16227 ts_tablet_manager.cc:1939] T 5e9734c77e2c4ca9b71fcf5eda783f26 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting consensus metadata
I20260925 05:15:12.241382 16184 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Bootstrap starting.
I20260925 05:15:12.241416 16182 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Bootstrap starting.
I20260925 05:15:12.241819 16184 tablet_bootstrap.cc:654] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:12.241818 16182 tablet_bootstrap.cc:654] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:12.242249 15662 catalog_manager.cc:4841] TS cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195): tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (table test-table [id=842527aa2aba49888e71a378539b2b66]) successfully deleted
I20260925 05:15:12.242537 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 56e9bc74147047499f28428877f58d34 (DEFAULT_TABLE table=test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:12.242658 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e9bc74147047499f28428877f58d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:12.242923 16182 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: No bootstrap required, opened a new log
I20260925 05:15:12.242969 16182 ts_tablet_manager.cc:1397] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:12.243132 16182 raft_consensus.cc:348] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.243199 16182 raft_consensus.cc:374] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:12.243218 16182 raft_consensus.cc:729] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Initialized, Role: FOLLOWER
I20260925 05:15:12.243250 16184 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: No bootstrap required, opened a new log
I20260925 05:15:12.243257 16182 consensus_queue.cc:260] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.243299 16184 ts_tablet_manager.cc:1397] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:12.243332 16182 ts_tablet_manager.cc:1428] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:12.243440 16184 raft_consensus.cc:348] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.243507 16184 raft_consensus.cc:374] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:12.243541 16184 raft_consensus.cc:729] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Initialized, Role: FOLLOWER
I20260925 05:15:12.243584 16184 consensus_queue.cc:260] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.243661 16184 ts_tablet_manager.cc:1428] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:12.244131 16183 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Bootstrap starting.
I20260925 05:15:12.244588 16183 tablet_bootstrap.cc:654] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:12.245204 16183 tablet_bootstrap.cc:492] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: No bootstrap required, opened a new log
I20260925 05:15:12.245260 16183 ts_tablet_manager.cc:1397] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:12.245415 16183 raft_consensus.cc:348] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.245499 16183 raft_consensus.cc:374] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:12.245514 16183 raft_consensus.cc:729] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f460f058ed30439eb57d5931debda31e, State: Initialized, Role: FOLLOWER
I20260925 05:15:12.245575 16183 consensus_queue.cc:260] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.245671 16183 ts_tablet_manager.cc:1428] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:12.245957 16226 tablet_replica.cc:330] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: stopping tablet replica
I20260925 05:15:12.246093 16226 raft_consensus.cc:2229] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:12.246176 16226 raft_consensus.cc:2258] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:12.246644 16226 ts_tablet_manager.cc:1905] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:12.247942 16226 ts_tablet_manager.cc:1918] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:12.248003 16226 log.cc:1196] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/5e9734c77e2c4ca9b71fcf5eda783f26
I20260925 05:15:12.248188 16226 ts_tablet_manager.cc:1939] T 5e9734c77e2c4ca9b71fcf5eda783f26 P f460f058ed30439eb57d5931debda31e: Deleting consensus metadata
I20260925 05:15:12.248610 15663 catalog_manager.cc:4841] TS f460f058ed30439eb57d5931debda31e (127.13.160.131:37587): tablet 5e9734c77e2c4ca9b71fcf5eda783f26 (table test-table [id=842527aa2aba49888e71a378539b2b66]) successfully deleted
I20260925 05:15:12.378242 16189 raft_consensus.cc:482] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:12.378348 16189 raft_consensus.cc:504] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.378674 16189 leader_election.cc:288] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:12.381716 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.379007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:47523 (local address 127.13.160.131:37587)
0925 05:15:12.379129 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.379131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.379140 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:12.379191 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.379192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.379204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.379298 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:12.379456 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.379963 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.380954 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.381340 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.381429 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.381438 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.381580 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.381620 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260925 05:15:12.381727 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.378957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:59773 (local address 127.13.160.129:40195)
0925 05:15:12.379008 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.379010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.379121 (+ 111us) server_negotiation.cc:407] Connection header received
0925 05:15:12.379193 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.379194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.379204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.379250 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:12.379422 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.379921 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.380974 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.381354 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.381442 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.381450 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.381613 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.381655 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:15:12.381760 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.378894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37587 (local address 127.13.160.130:47523)
0925 05:15:12.379055 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.379067 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.379128 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.379301 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.379302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.379304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:12.379365 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.379367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.380062 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.380063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.380854 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.380856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.381426 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.381427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.381486 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.381489 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.381572 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.381612 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":30,"thread_start_us":58,"threads_started":1}
I20260925 05:15:12.381762 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.378869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.13.160.130:59773)
0925 05:15:12.379034 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.379047 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.379133 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.379263 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.379264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.379266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:12.379334 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.379336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.380015 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.380016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.380858 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.380860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.381441 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.381442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.381512 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.381515 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.381605 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.381646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":23,"thread_start_us":83,"threads_started":1}
I20260925 05:15:12.382002 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e9bc74147047499f28428877f58d34" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e" is_pre_election: true
I20260925 05:15:12.382133 16113 raft_consensus.cc:2454] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:12.382094 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e9bc74147047499f28428877f58d34" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" is_pre_election: true
I20260925 05:15:12.382223 15833 raft_consensus.cc:2454] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:12.382340 15907 leader_election.cc:302] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:12.382477 16189 raft_consensus.cc:2781] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:12.382531 16189 raft_consensus.cc:482] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:12.382546 16189 raft_consensus.cc:3037] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.383249 16189 raft_consensus.cc:504] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.383394 16189 leader_election.cc:288] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Requested vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:12.383533 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e9bc74147047499f28428877f58d34" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
I20260925 05:15:12.383541 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e9bc74147047499f28428877f58d34" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e"
I20260925 05:15:12.383610 15833 raft_consensus.cc:3037] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.383635 16113 raft_consensus.cc:3037] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:12.384243 16113 raft_consensus.cc:2454] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:12.384358 15833 raft_consensus.cc:2454] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:12.384529 15906 leader_election.cc:302] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, cdb99d2b1e7b4d5a87a38fb344c81118; no voters:
I20260925 05:15:12.384632 16189 raft_consensus.cc:2781] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:12.384773 16189 raft_consensus.cc:686] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Becoming Leader. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Running, Role: LEADER
I20260925 05:15:12.384855 16189 consensus_queue.cc:237] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:12.385613 15675 catalog_manager.cc:5495] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "3b13135cca2449f990db3b1a9a8c9f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:12.394248 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.391670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.13.160.130:38221)
0925 05:15:12.391839 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.391841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.391847 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:12.391878 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.391879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.391897 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.391989 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.392102 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.392643 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.393391 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.393621 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.393707 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.393870 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.393986 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.394075 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.394098 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.394120 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260925 05:15:12.399544 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.396749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.13.160.130:38021)
0925 05:15:12.396924 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.396934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.397021 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.397210 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.397211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.397212 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:12.397277 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.397278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.397943 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.397944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.398620 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.398622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.399058 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.399059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.399124 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.399146 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.399332 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.399334 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.399385 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.399425 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260925 05:15:12.399530 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.396846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:38021 (local address 127.13.160.129:40195)
0925 05:15:12.396894 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.396897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.397003 (+ 106us) server_negotiation.cc:407] Connection header received
0925 05:15:12.397121 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.397122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.397136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.397208 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.397360 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.397859 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.398710 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.398968 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.399058 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.399240 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.399273 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.399323 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.399391 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.399417 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:12.401721 16113 raft_consensus.cc:1261] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:12.401793 15833 raft_consensus.cc:1261] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:12.402016 16234 consensus_queue.cc:1035] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.402132 16189 consensus_queue.cc:1035] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:12.409490 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.407189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:54035 (local address 127.13.160.130:38221)
0925 05:15:12.407230 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:12.407232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:12.407241 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:12.407299 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:12.407300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:12.407312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:12.407365 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:12.407528 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.408002 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.408728 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.408939 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.409014 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.409234 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.409260 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.409323 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:12.409384 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 05:15:12.409404 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:15:12.409492 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:12.407124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38221 (local address 127.13.160.129:54035)
0925 05:15:12.407183 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:12.407195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:12.407237 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:12.407368 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:12.407369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:12.407370 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:12.407416 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.407418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.408086 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.408087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.408642 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:12.408644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:12.409012 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:12.409013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:12.409067 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:12.409073 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:12.409323 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:12.409325 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:12.409379 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 05:15:12.409426 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 05:15:12.409941 16205 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 05:15:13.932126 15675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:13.932233 15675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:13.933024 15675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Sending DeleteTablet for 3 replicas of tablet 56e9bc74147047499f28428877f58d34
I20260925 05:15:13.933312 15813 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e9bc74147047499f28428877f58d34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:13 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55690
I20260925 05:15:13.933321 15953 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e9bc74147047499f28428877f58d34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:13 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37284
I20260925 05:15:13.933537 16266 tablet_replica.cc:330] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: stopping tablet replica
I20260925 05:15:13.933526 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e9bc74147047499f28428877f58d34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:13 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37756
I20260925 05:15:13.933640 16266 raft_consensus.cc:2229] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:13.933681 16266 raft_consensus.cc:2258] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:13.933790 16268 tablet_replica.cc:330] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: stopping tablet replica
I20260925 05:15:13.933864 16268 raft_consensus.cc:2229] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:13.933909 16268 raft_consensus.cc:2258] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:13.933921 16266 ts_tablet_manager.cc:1905] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:13.934132 16268 ts_tablet_manager.cc:1905] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:13.935936 16266 ts_tablet_manager.cc:1918] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 05:15:13.936039 16268 ts_tablet_manager.cc:1918] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 05:15:13.936065 16266 log.cc:1196] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/56e9bc74147047499f28428877f58d34
I20260925 05:15:13.936194 16268 log.cc:1196] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/56e9bc74147047499f28428877f58d34
I20260925 05:15:13.936305 16266 ts_tablet_manager.cc:1939] T 56e9bc74147047499f28428877f58d34 P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting consensus metadata
I20260925 05:15:13.936394 16267 tablet_replica.cc:330] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: stopping tablet replica
I20260925 05:15:13.936461 16267 raft_consensus.cc:2229] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:13.936594 16267 raft_consensus.cc:2258] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:13.936750 15662 catalog_manager.cc:4841] TS cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195): tablet 56e9bc74147047499f28428877f58d34 (table test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]) successfully deleted
I20260925 05:15:13.936880 16267 ts_tablet_manager.cc:1905] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:13.937896 16268 ts_tablet_manager.cc:1939] T 56e9bc74147047499f28428877f58d34 P f460f058ed30439eb57d5931debda31e: Deleting consensus metadata
I20260925 05:15:13.938128 16267 ts_tablet_manager.cc:1918] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 05:15:13.938261 16267 log.cc:1196] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/56e9bc74147047499f28428877f58d34
I20260925 05:15:13.938408 15663 catalog_manager.cc:4841] TS f460f058ed30439eb57d5931debda31e (127.13.160.131:37587): tablet 56e9bc74147047499f28428877f58d34 (table test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]) successfully deleted
I20260925 05:15:13.938570 16267 ts_tablet_manager.cc:1939] T 56e9bc74147047499f28428877f58d34 P 3b13135cca2449f990db3b1a9a8c9f91: Deleting consensus metadata
I20260925 05:15:13.938957 15662 catalog_manager.cc:4841] TS 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221): tablet 56e9bc74147047499f28428877f58d34 (table test-table [id=ad492c65ec3e4b8bbb54489ea4222a9b]) successfully deleted
Sep 25 05:15:13 dist-test-slave-9tgz krb5kdc[15631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313313, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 05:15:13 dist-test-slave-9tgz krb5kdc[15631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313313, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:13.966362 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:13.950465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.13.160.190:40279)
0925 05:15:13.950611 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:13.950614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:13.950624 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:13.950649 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:13.950649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:13.950660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:13.950729 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:13.951345 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:13.951850 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:13.952550 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:13.952741 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:13.952791 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:13.964034 (+ 11243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:13.964036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:13.964036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:13.964045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:13.965403 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:13.965727 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:13.965728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:13.965728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:13.965782 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:13.965944 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:13.965944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:13.965945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:13.966057 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:13.966142 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:13.966222 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 05:15:13.966288 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260925 05:15:13.967844 15675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:13.968062 15675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:13.969372 15953 tablet_service.cc:1467] Processing CreateTablet for tablet 397480979f9545cd9253809aa66f070e (DEFAULT_TABLE table=test-table [id=f371446acd124e1ba7ce331ab801e4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:13.969389 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 397480979f9545cd9253809aa66f070e (DEFAULT_TABLE table=test-table [id=f371446acd124e1ba7ce331ab801e4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:13.969414 15813 tablet_service.cc:1467] Processing CreateTablet for tablet 397480979f9545cd9253809aa66f070e (DEFAULT_TABLE table=test-table [id=f371446acd124e1ba7ce331ab801e4d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:13.969518 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397480979f9545cd9253809aa66f070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:13.969517 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397480979f9545cd9253809aa66f070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:13.969564 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397480979f9545cd9253809aa66f070e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:13.971166 16278 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Bootstrap starting.
I20260925 05:15:13.971516 16278 tablet_bootstrap.cc:654] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:13.971853 16277 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Bootstrap starting.
I20260925 05:15:13.972231 16277 tablet_bootstrap.cc:654] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:13.972680 16278 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: No bootstrap required, opened a new log
I20260925 05:15:13.972746 16278 ts_tablet_manager.cc:1397] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:13.972818 16277 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: No bootstrap required, opened a new log
I20260925 05:15:13.972873 16277 ts_tablet_manager.cc:1397] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:13.972885 16278 raft_consensus.cc:348] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.972965 16278 raft_consensus.cc:374] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:13.972996 16278 raft_consensus.cc:729] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Initialized, Role: FOLLOWER
I20260925 05:15:13.972997 16277 raft_consensus.cc:348] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.973044 16278 consensus_queue.cc:260] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.973075 16277 raft_consensus.cc:374] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:13.973101 16277 raft_consensus.cc:729] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Initialized, Role: FOLLOWER
I20260925 05:15:13.973145 16277 consensus_queue.cc:260] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.973241 16277 ts_tablet_manager.cc:1428] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:13.973333 16278 ts_tablet_manager.cc:1428] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:13.973456 16279 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Bootstrap starting.
I20260925 05:15:13.973809 16279 tablet_bootstrap.cc:654] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:13.974398 16279 tablet_bootstrap.cc:492] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: No bootstrap required, opened a new log
I20260925 05:15:13.974442 16279 ts_tablet_manager.cc:1397] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:13.974612 16279 raft_consensus.cc:348] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.974671 16279 raft_consensus.cc:374] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:13.974712 16279 raft_consensus.cc:729] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f460f058ed30439eb57d5931debda31e, State: Initialized, Role: FOLLOWER
I20260925 05:15:13.974769 16279 consensus_queue.cc:260] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:13.974960 16279 ts_tablet_manager.cc:1428] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:14.105728 16261 raft_consensus.cc:482] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:14.105835 16261 raft_consensus.cc:504] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.106033 16261 leader_election.cc:288] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:14.106328 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397480979f9545cd9253809aa66f070e" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e" is_pre_election: true
I20260925 05:15:14.106328 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397480979f9545cd9253809aa66f070e" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" is_pre_election: true
I20260925 05:15:14.106437 16113 raft_consensus.cc:2454] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:14.106437 15833 raft_consensus.cc:2454] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:14.106618 15906 leader_election.cc:302] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, cdb99d2b1e7b4d5a87a38fb344c81118; no voters:
I20260925 05:15:14.106710 16261 raft_consensus.cc:2781] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:14.106762 16261 raft_consensus.cc:482] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:14.106796 16261 raft_consensus.cc:3037] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.107645 16261 raft_consensus.cc:504] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.107798 16261 leader_election.cc:288] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Requested vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:14.107954 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397480979f9545cd9253809aa66f070e" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
I20260925 05:15:14.107961 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397480979f9545cd9253809aa66f070e" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e"
I20260925 05:15:14.108024 16113 raft_consensus.cc:3037] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.108043 15833 raft_consensus.cc:3037] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.108644 16113 raft_consensus.cc:2454] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:14.108707 15833 raft_consensus.cc:2454] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:14.108837 15907 leader_election.cc:302] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:14.108934 16261 raft_consensus.cc:2781] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:14.109002 16261 raft_consensus.cc:686] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Becoming Leader. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Running, Role: LEADER
I20260925 05:15:14.109063 16261 consensus_queue.cc:237] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.109717 15676 catalog_manager.cc:5495] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "3b13135cca2449f990db3b1a9a8c9f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:14.120692 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:14.118413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.13.160.130:38221)
0925 05:15:14.118579 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:14.118581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:14.118587 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:14.118606 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:14.118606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:14.118616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:14.118668 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:14.118789 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:14.119332 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:14.120057 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:14.120241 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:14.120347 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:14.120455 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:14.120480 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:14.120515 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:14.120577 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 05:15:14.120593 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260925 05:15:14.122864 15833 raft_consensus.cc:1261] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:14.123047 16113 raft_consensus.cc:1261] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:14.123092 16261 consensus_queue.cc:1035] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:14.123220 16257 consensus_queue.cc:1035] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:14.133260 15675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:14.133360 15675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:14.133967 15675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Sending DeleteTablet for 3 replicas of tablet 397480979f9545cd9253809aa66f070e
I20260925 05:15:14.134228 15813 tablet_service.cc:1514] Processing DeleteTablet for tablet 397480979f9545cd9253809aa66f070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:14 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55690
I20260925 05:15:14.134284 15953 tablet_service.cc:1514] Processing DeleteTablet for tablet 397480979f9545cd9253809aa66f070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:14 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37284
I20260925 05:15:14.134342 16266 tablet_replica.cc:330] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: stopping tablet replica
I20260925 05:15:14.134342 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet 397480979f9545cd9253809aa66f070e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:14 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37756
I20260925 05:15:14.134400 16266 raft_consensus.cc:2229] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:14.134428 16268 tablet_replica.cc:330] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: stopping tablet replica
I20260925 05:15:14.134450 16266 raft_consensus.cc:2258] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:14.134449 16267 tablet_replica.cc:330] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: stopping tablet replica
I20260925 05:15:14.134475 16268 raft_consensus.cc:2229] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:14.134514 16267 raft_consensus.cc:2229] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:14.134542 16268 raft_consensus.cc:2258] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:14.134567 16267 raft_consensus.cc:2258] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:14.134742 16266 ts_tablet_manager.cc:1905] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:14.134883 16267 ts_tablet_manager.cc:1905] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:14.135057 15675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:14.135193 15675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:14.136462 15953 tablet_service.cc:1467] Processing CreateTablet for tablet e1984c7bbe714102ae2f291fad1d7aac (DEFAULT_TABLE table=test-table [id=3254867f632c4dfbbe2964dbc22f05bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:14.136482 16093 tablet_service.cc:1467] Processing CreateTablet for tablet e1984c7bbe714102ae2f291fad1d7aac (DEFAULT_TABLE table=test-table [id=3254867f632c4dfbbe2964dbc22f05bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:14.136611 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1984c7bbe714102ae2f291fad1d7aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:14.136700 16267 ts_tablet_manager.cc:1918] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:14.136801 16267 log.cc:1196] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/397480979f9545cd9253809aa66f070e
I20260925 05:15:14.137049 16267 ts_tablet_manager.cc:1939] T 397480979f9545cd9253809aa66f070e P 3b13135cca2449f990db3b1a9a8c9f91: Deleting consensus metadata
I20260925 05:15:14.137197 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1984c7bbe714102ae2f291fad1d7aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:14.138052 16279 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Bootstrap starting.
I20260925 05:15:14.138095 15813 tablet_service.cc:1467] Processing CreateTablet for tablet e1984c7bbe714102ae2f291fad1d7aac (DEFAULT_TABLE table=test-table [id=3254867f632c4dfbbe2964dbc22f05bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:14.138409 16277 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Bootstrap starting.
I20260925 05:15:14.138684 16279 tablet_bootstrap.cc:654] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:14.138689 16266 ts_tablet_manager.cc:1918] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:14.138741 16266 log.cc:1196] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/397480979f9545cd9253809aa66f070e
I20260925 05:15:14.138747 16277 tablet_bootstrap.cc:654] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:14.138949 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1984c7bbe714102ae2f291fad1d7aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:14.139709 16277 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: No bootstrap required, opened a new log
I20260925 05:15:14.139766 16277 ts_tablet_manager.cc:1397] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:14.139882 16277 raft_consensus.cc:348] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.139972 16277 raft_consensus.cc:374] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:14.140002 16277 raft_consensus.cc:729] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Initialized, Role: FOLLOWER
I20260925 05:15:14.140048 16277 consensus_queue.cc:260] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.140146 16277 ts_tablet_manager.cc:1428] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:14.140278 16278 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118: Bootstrap starting.
I20260925 05:15:14.140334 16266 ts_tablet_manager.cc:1939] T 397480979f9545cd9253809aa66f070e P cdb99d2b1e7b4d5a87a38fb344c81118: Deleting consensus metadata
I20260925 05:15:14.140746 16278 tablet_bootstrap.cc:654] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:14.140772 15662 catalog_manager.cc:4841] TS 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221): tablet 397480979f9545cd9253809aa66f070e (table test-table [id=f371446acd124e1ba7ce331ab801e4d3]) successfully deleted
I20260925 05:15:14.140945 16268 ts_tablet_manager.cc:1905] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:14.141049 15662 catalog_manager.cc:4841] TS cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195): tablet 397480979f9545cd9253809aa66f070e (table test-table [id=f371446acd124e1ba7ce331ab801e4d3]) successfully deleted
I20260925 05:15:14.141372 16278 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118: No bootstrap required, opened a new log
I20260925 05:15:14.141445 16278 ts_tablet_manager.cc:1397] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:14.141562 16278 raft_consensus.cc:348] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.141610 16278 raft_consensus.cc:374] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:14.141624 16278 raft_consensus.cc:729] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb99d2b1e7b4d5a87a38fb344c81118, State: Initialized, Role: FOLLOWER
I20260925 05:15:14.141690 16278 consensus_queue.cc:260] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.141779 16278 ts_tablet_manager.cc:1428] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:14.142660 16268 ts_tablet_manager.cc:1918] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:14.142719 16268 log.cc:1196] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/397480979f9545cd9253809aa66f070e
I20260925 05:15:14.142925 16268 ts_tablet_manager.cc:1939] T 397480979f9545cd9253809aa66f070e P f460f058ed30439eb57d5931debda31e: Deleting consensus metadata
I20260925 05:15:14.143296 15663 catalog_manager.cc:4841] TS f460f058ed30439eb57d5931debda31e (127.13.160.131:37587): tablet 397480979f9545cd9253809aa66f070e (table test-table [id=f371446acd124e1ba7ce331ab801e4d3]) successfully deleted
I20260925 05:15:14.143532 16279 tablet_bootstrap.cc:492] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: No bootstrap required, opened a new log
I20260925 05:15:14.143565 16279 ts_tablet_manager.cc:1397] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260925 05:15:14.143673 16279 raft_consensus.cc:348] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.143718 16279 raft_consensus.cc:374] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:14.143729 16279 raft_consensus.cc:729] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f460f058ed30439eb57d5931debda31e, State: Initialized, Role: FOLLOWER
I20260925 05:15:14.143791 16279 consensus_queue.cc:260] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.143903 16279 ts_tablet_manager.cc:1428] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:14.286181 16261 raft_consensus.cc:482] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:14.286298 16261 raft_consensus.cc:504] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.286489 16261 leader_election.cc:288] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:14.286688 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1984c7bbe714102ae2f291fad1d7aac" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" is_pre_election: true
I20260925 05:15:14.286791 15833 raft_consensus.cc:2454] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:14.286746 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1984c7bbe714102ae2f291fad1d7aac" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e" is_pre_election: true
I20260925 05:15:14.286872 16113 raft_consensus.cc:2454] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 0.
I20260925 05:15:14.287034 15906 leader_election.cc:302] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, cdb99d2b1e7b4d5a87a38fb344c81118; no voters:
I20260925 05:15:14.287133 16261 raft_consensus.cc:2781] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:14.287173 16261 raft_consensus.cc:482] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:14.287195 16261 raft_consensus.cc:3037] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.287827 16261 raft_consensus.cc:504] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.287972 16261 leader_election.cc:288] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Requested vote from peers cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195), f460f058ed30439eb57d5931debda31e (127.13.160.131:37587)
I20260925 05:15:14.288115 15833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1984c7bbe714102ae2f291fad1d7aac" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118"
I20260925 05:15:14.288201 15833 raft_consensus.cc:3037] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.288188 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1984c7bbe714102ae2f291fad1d7aac" candidate_uuid: "3b13135cca2449f990db3b1a9a8c9f91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f460f058ed30439eb57d5931debda31e"
I20260925 05:15:14.288266 16113 raft_consensus.cc:3037] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:14.289007 16113 raft_consensus.cc:2454] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:14.289007 15833 raft_consensus.cc:2454] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b13135cca2449f990db3b1a9a8c9f91 in term 1.
I20260925 05:15:14.289242 15907 leader_election.cc:302] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b13135cca2449f990db3b1a9a8c9f91, f460f058ed30439eb57d5931debda31e; no voters:
I20260925 05:15:14.289348 16261 raft_consensus.cc:2781] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:14.289423 16261 raft_consensus.cc:686] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Becoming Leader. State: Replica: 3b13135cca2449f990db3b1a9a8c9f91, State: Running, Role: LEADER
I20260925 05:15:14.289482 16261 consensus_queue.cc:237] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } }
I20260925 05:15:14.290091 15675 catalog_manager.cc:5495] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "3b13135cca2449f990db3b1a9a8c9f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b13135cca2449f990db3b1a9a8c9f91" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:14.327332 16113 raft_consensus.cc:1261] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:14.327397 15833 raft_consensus.cc:1261] T e1984c7bbe714102ae2f291fad1d7aac P cdb99d2b1e7b4d5a87a38fb344c81118 [term 1 FOLLOWER]: Refusing update from remote peer 3b13135cca2449f990db3b1a9a8c9f91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:14.327592 16244 consensus_queue.cc:1035] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f460f058ed30439eb57d5931debda31e" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:14.327689 16261 consensus_queue.cc:1035] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb99d2b1e7b4d5a87a38fb344c81118" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 05:15:14.331691 16248 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 05:15:15.848618 15675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:15.848716 15675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:15.849596 15675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613fc87160a64a89b5d7811bc9cca52e: Sending DeleteTablet for 3 replicas of tablet e1984c7bbe714102ae2f291fad1d7aac
I20260925 05:15:15.849844 15813 tablet_service.cc:1514] Processing DeleteTablet for tablet e1984c7bbe714102ae2f291fad1d7aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:15 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55690
I20260925 05:15:15.849869 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet e1984c7bbe714102ae2f291fad1d7aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:15 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37756
I20260925 05:15:15.849943 15953 tablet_service.cc:1514] Processing DeleteTablet for tablet e1984c7bbe714102ae2f291fad1d7aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:15 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:37284
I20260925 05:15:15.850015 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15748
I20260925 05:15:15.850667 16320 tablet_replica.cc:330] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: stopping tablet replica
I20260925 05:15:15.850745 16320 raft_consensus.cc:2229] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:15.850816 16320 raft_consensus.cc:2258] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:15.851085 16320 ts_tablet_manager.cc:1905] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:15.852802 16320 ts_tablet_manager.cc:1918] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 05:15:15.852874 16320 log.cc:1196] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/e1984c7bbe714102ae2f291fad1d7aac
I20260925 05:15:15.853124 16320 ts_tablet_manager.cc:1939] T e1984c7bbe714102ae2f291fad1d7aac P 3b13135cca2449f990db3b1a9a8c9f91: Deleting consensus metadata
I20260925 05:15:15.853434 15662 catalog_manager.cc:4841] TS 3b13135cca2449f990db3b1a9a8c9f91 (127.13.160.130:38221): tablet e1984c7bbe714102ae2f291fad1d7aac (table test-table [id=3254867f632c4dfbbe2964dbc22f05bf]) successfully deleted
I20260925 05:15:15.855051 16319 tablet_replica.cc:330] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: stopping tablet replica
I20260925 05:15:15.855139 16319 raft_consensus.cc:2229] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:15.855189 16319 raft_consensus.cc:2258] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:15.855424 16319 ts_tablet_manager.cc:1905] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:15.856722 16319 ts_tablet_manager.cc:1918] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 05:15:15.856794 16319 log.cc:1196] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/e1984c7bbe714102ae2f291fad1d7aac
I20260925 05:15:15.857002 16319 ts_tablet_manager.cc:1939] T e1984c7bbe714102ae2f291fad1d7aac P f460f058ed30439eb57d5931debda31e: Deleting consensus metadata
W20260925 05:15:15.857563 15662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:40195 (error 108)
I20260925 05:15:15.857599 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15888
I20260925 05:15:15.858512 15663 catalog_manager.cc:4841] TS f460f058ed30439eb57d5931debda31e (127.13.160.131:37587): tablet e1984c7bbe714102ae2f291fad1d7aac (table test-table [id=3254867f632c4dfbbe2964dbc22f05bf]) successfully deleted
I20260925 05:15:15.858765 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:15.858448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40195 (local address 127.0.0.1:35216)
0925 05:15:15.858663 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:15.858717 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":107,"threads_started":1}
W20260925 05:15:15.858858 15662 catalog_manager.cc:4584] TS cdb99d2b1e7b4d5a87a38fb344c81118 (127.13.160.129:40195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1984c7bbe714102ae2f291fad1d7aac: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40195: connect: Connection refused (error 111)
I20260925 05:15:15.864362 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16028
I20260925 05:15:15.868876 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 15644
2026-09-25T05:15:15Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6161 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:15 dist-test-slave-9tgz krb5kdc[16325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:15 dist-test-slave-9tgz krb5kdc[16325](info): set up 2 sockets
Sep 25 05:15:15 dist-test-slave-9tgz krb5kdc[16325](info): commencing operation
krb5kdc: starting...
W20260925 05:15:16.911550 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:16 dist-test-slave-9tgz krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:17.033936 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45477
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:45477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:17.118327 16338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:17.118625 16338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:17.118670 16338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:17.119947 16338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:17.120029 16338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:17.120057 16338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:17.120091 16338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:17.120126 16338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:17.121672 16338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:45477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:17.121977 16338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:17.122237 16338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:17.127720 16338 server_base.cc:1034] running on GCE node
W20260925 05:15:17.127696 16346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.127696 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.127683 16343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:17.128273 16338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:17.128634 16338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:17.129809 16338 hybrid_clock.cc:648] HybridClock initialized: now 1790313317129795 us; error 44 us; skew 500 ppm
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:17.132400 16338 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:17.132885 16338 webserver.cc:460] Webserver started at http://127.13.160.190:36605/ using document root <none> and password file <none>
I20260925 05:15:17.133086 16338 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:17.133129 16338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:17.133220 16338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:17.134244 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "193e6e61abc94456a16517da52ec8ae6"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "41176815b2647f464720f2fe48e6c2e1"
server_key_iv: "415f3388cf09499ede20587c82bf643f"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.134547 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "193e6e61abc94456a16517da52ec8ae6"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "41176815b2647f464720f2fe48e6c2e1"
server_key_iv: "415f3388cf09499ede20587c82bf643f"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.135906 16338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:17.136620 16353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.136921 16338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:15:17.137017 16338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "193e6e61abc94456a16517da52ec8ae6"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "41176815b2647f464720f2fe48e6c2e1"
server_key_iv: "415f3388cf09499ede20587c82bf643f"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.137094 16338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:17.153015 16338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:17.155362 16338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:17.155519 16338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:17.159873 16338 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:45477
I20260925 05:15:17.159902 16405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:45477 every 8 connection(s)
I20260925 05:15:17.160282 16338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:17.161149 16406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:17.163529 16406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6: Bootstrap starting.
I20260925 05:15:17.164130 16406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:17.164407 16406 log.cc:824] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:17.165096 16406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6: No bootstrap required, opened a new log
I20260925 05:15:17.166692 16406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } }
I20260925 05:15:17.166857 16406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:17.166934 16406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193e6e61abc94456a16517da52ec8ae6, State: Initialized, Role: FOLLOWER
I20260925 05:15:17.167119 16406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } }
I20260925 05:15:17.167217 16406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:17.167258 16406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:17.167312 16406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:17.167845 16406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } }
I20260925 05:15:17.167959 16406 leader_election.cc:302] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 193e6e61abc94456a16517da52ec8ae6; no voters:
I20260925 05:15:17.168143 16406 leader_election.cc:288] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:17.168186 16409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:17.168387 16409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [term 1 LEADER]: Becoming Leader. State: Replica: 193e6e61abc94456a16517da52ec8ae6, State: Running, Role: LEADER
I20260925 05:15:17.168403 16406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:17.168483 16409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } }
I20260925 05:15:17.168967 16410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "193e6e61abc94456a16517da52ec8ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } } }
I20260925 05:15:17.169062 16410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:17.169147 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16338
I20260925 05:15:17.169237 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:17.169418 13954 external_mini_cluster.cc:1428] Setting key 6b3d423f984e556c6d0ad8d462cce8cb
I20260925 05:15:17.168951 16411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 193e6e61abc94456a16517da52ec8ae6. Latest consensus state: current_term: 1 leader_uuid: "193e6e61abc94456a16517da52ec8ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "193e6e61abc94456a16517da52ec8ae6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45477 } } }
I20260925 05:15:17.169569 16411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:17.169694 16413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:17.170226 16413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:17.175366 16413 catalog_manager.cc:1347] Generated new cluster ID: 6fe98b3f5660498c8e2da61b6be0ea03
I20260925 05:15:17.175444 16413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:17.180222 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.169923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35246 (local address 127.13.160.190:45477)
0925 05:15:17.170108 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:17.170119 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:17.171112 (+ 993us) server_negotiation.cc:407] Connection header received
0925 05:15:17.171276 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:17.171284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:17.171346 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:17.171510 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:17.172127 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.173064 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.173842 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.174280 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.174395 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.176887 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:17.176892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:17.176892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:17.176905 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:17.178616 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.179062 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.179072 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.179072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.179164 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.179370 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.179371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.179371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.179629 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:17.179794 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:17.179876 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 05:15:17.180019 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260925 05:15:17.185674 16413 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:17.186378 16413 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:17.191509 16413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 193e6e61abc94456a16517da52ec8ae6: Generated new TSK 0
I20260925 05:15:17.191725 16413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:17.222543 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:17.306389 16432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:17.306615 16432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:17.306643 16432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:17.307957 16432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:17.308004 16432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:17.308034 16432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:17.309545 16432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:17.309799 16432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:17.310040 16432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:17.315353 16432 server_base.cc:1034] running on GCE node
W20260925 05:15:17.315317 16437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.315316 16440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.315327 16438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:17.315752 16432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:17.315996 16432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:17.317157 16432 hybrid_clock.cc:648] HybridClock initialized: now 1790313317317154 us; error 33 us; skew 500 ppm
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:17.319823 16432 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:17.320302 16432 webserver.cc:460] Webserver started at http://127.13.160.129:41643/ using document root <none> and password file <none>
I20260925 05:15:17.320531 16432 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:17.320597 16432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:17.320748 16432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:17.321805 16432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "3ef0d4930e84445cbab4887d3121aabb"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "702301873defeb1f1a23148a8e45a308"
server_key_iv: "309d6a2ca10fe2c61e6c146fd47d112e"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.322172 16432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "3ef0d4930e84445cbab4887d3121aabb"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "702301873defeb1f1a23148a8e45a308"
server_key_iv: "309d6a2ca10fe2c61e6c146fd47d112e"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.323653 16432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:17.324350 16447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.324527 16432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.324620 16432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "3ef0d4930e84445cbab4887d3121aabb"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "702301873defeb1f1a23148a8e45a308"
server_key_iv: "309d6a2ca10fe2c61e6c146fd47d112e"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.324697 16432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:17.357023 16432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:17.359496 16432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:17.359668 16432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:17.359920 16432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:17.360224 16432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:17.360275 16432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.360316 16432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:17.360347 16432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.366025 16432 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:44551
I20260925 05:15:17.366077 16560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:44551 every 8 connection(s)
I20260925 05:15:17.366391 16432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:17.367902 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16432
I20260925 05:15:17.368000 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:17.368177 13954 external_mini_cluster.cc:1428] Setting key 5a092bad17c5c13530093ea0a46f8922
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
I20260925 05:15:17.383522 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.367124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:46983 (local address 127.13.160.190:45477)
0925 05:15:17.367167 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:17.367170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:17.367370 (+ 200us) server_negotiation.cc:407] Connection header received
0925 05:15:17.367955 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:17.367957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:17.367974 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:17.368040 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:17.369280 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.369919 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.371109 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.371325 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.371392 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.374492 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:17.374494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:17.374494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:17.374504 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:17.381731 (+ 7227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.382375 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.382377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.382377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.382442 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.382730 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.382730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.382731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.382870 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:17.382937 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:17.383320 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 05:15:17.383422 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:15:17.384171 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.367090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45477 (local address 127.13.160.129:46983)
0925 05:15:17.367248 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:17.367286 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:17.367854 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:17.368161 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:17.368165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:17.368965 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:17.369177 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.369182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.370012 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.370013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.371006 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.371009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.371418 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.371419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.371587 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.372279 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:17.372288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:17.374346 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:17.381871 (+ 7525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.381889 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.381892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.382277 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.382514 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.382515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.382515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.382661 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.382983 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:17.382991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:17.383171 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 05:15:17.383688 (+ 517us) client_negotiation.cc:240] Negotiation successful
0925 05:15:17.383895 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260925 05:15:17.386413 16561 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45477
I20260925 05:15:17.386549 16561 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:17.386791 16561 heartbeater.cc:511] Master 127.13.160.190:45477 requested a full tablet report, sending...
I20260925 05:15:17.387506 16370 ts_manager.cc:194] Registered new tserver with Master: 3ef0d4930e84445cbab4887d3121aabb (127.13.160.129:44551)
I20260925 05:15:17.388510 16370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:46983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:17.397995 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:17.484220 16566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:17.484459 16566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:17.484488 16566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:17.485713 16566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:17.485757 16566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:17.485786 16566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:17.487375 16566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:17.487609 16566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:17.487820 16566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:17.493040 16574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.493017 16571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:17.493053 16566 server_base.cc:1034] running on GCE node
W20260925 05:15:17.493017 16572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:17.493541 16566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:17.493816 16566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:17.494966 16566 hybrid_clock.cc:648] HybridClock initialized: now 1790313317494951 us; error 33 us; skew 500 ppm
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:17.497507 16566 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:17.498003 16566 webserver.cc:460] Webserver started at http://127.13.160.130:35821/ using document root <none> and password file <none>
I20260925 05:15:17.498263 16566 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:17.498312 16566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:17.498482 16566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:17.499680 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "bcfe295250e244fa8d81ba0172ffb6a1"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "48c61a89236bf6df1954b308a8a2e2e1"
server_key_iv: "50d3f777a10453f3d3bc6aa0c24ff139"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.500056 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "bcfe295250e244fa8d81ba0172ffb6a1"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "48c61a89236bf6df1954b308a8a2e2e1"
server_key_iv: "50d3f777a10453f3d3bc6aa0c24ff139"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.501507 16566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:17.502305 16581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.502517 16566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:17.502609 16566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "bcfe295250e244fa8d81ba0172ffb6a1"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "48c61a89236bf6df1954b308a8a2e2e1"
server_key_iv: "50d3f777a10453f3d3bc6aa0c24ff139"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.502708 16566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:17.523264 16566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:17.525444 16566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:17.525622 16566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:17.525882 16566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:17.526216 16566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:17.526249 16566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.526273 16566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:17.526316 16566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.532356 16566 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:37721
I20260925 05:15:17.532398 16694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:37721 every 8 connection(s)
I20260925 05:15:17.532756 16566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:17.533267 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16566
I20260925 05:15:17.533366 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:17.533535 13954 external_mini_cluster.cc:1428] Setting key 62ec30a30941dcf5337e99228288c8cb
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
I20260925 05:15:17.544471 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.534133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:43525 (local address 127.13.160.190:45477)
0925 05:15:17.534196 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:17.534199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:17.534435 (+ 236us) server_negotiation.cc:407] Connection header received
0925 05:15:17.535003 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:17.535005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:17.535019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:17.535085 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:17.536037 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.536826 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.538452 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.538642 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.538701 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.541401 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:17.541403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:17.541403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:17.541412 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:17.542988 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.543413 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.543414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.543415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.543475 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.543743 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.543744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.543745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.543897 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:17.543978 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:17.544282 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 05:15:17.544376 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 05:15:17.544703 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.534131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45477 (local address 127.13.160.130:43525)
0925 05:15:17.534319 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:17.534361 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:17.534852 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:17.535147 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:17.535150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:17.535730 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:17.535939 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.535944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.536938 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.536939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.538367 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.538371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.538738 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.538739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.538923 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.539569 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:17.539582 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:17.541292 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:17.543079 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.543087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.543090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.543332 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.543541 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.543542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.543542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.543659 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.543978 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:17.543986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:17.544141 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 05:15:17.544251 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 05:15:17.544428 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
I20260925 05:15:17.545287 16695 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45477
I20260925 05:15:17.545397 16695 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:17.545617 16695 heartbeater.cc:511] Master 127.13.160.190:45477 requested a full tablet report, sending...
I20260925 05:15:17.546065 16370 ts_manager.cc:194] Registered new tserver with Master: bcfe295250e244fa8d81ba0172ffb6a1 (127.13.160.130:37721)
I20260925 05:15:17.546586 16370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:43525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:17.561787 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:17.645133 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:17.645437 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:17.645478 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:17.646786 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:17.646862 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:17.646932 16700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:17.648492 16700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:17.648792 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:17.649046 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:17.654081 16706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.654085 16705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:17.654085 16708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:17.654392 16700 server_base.cc:1034] running on GCE node
I20260925 05:15:17.654642 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:17.654937 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:17.656103 16700 hybrid_clock.cc:648] HybridClock initialized: now 1790313317656095 us; error 33 us; skew 500 ppm
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:17.658823 16700 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:17.659330 16700 webserver.cc:460] Webserver started at http://127.13.160.131:33529/ using document root <none> and password file <none>
I20260925 05:15:17.659582 16700 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:17.659652 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:17.659803 16700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:17.660908 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "9ca95638f0194c53aef44fc1c1d3b1dc"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "83f1c9c40dfc25e67d127eb049142ca8"
server_key_iv: "e2dc1bb88ea94ecadd07c0b077cfe7ae"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.661279 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "9ca95638f0194c53aef44fc1c1d3b1dc"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "83f1c9c40dfc25e67d127eb049142ca8"
server_key_iv: "e2dc1bb88ea94ecadd07c0b077cfe7ae"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.662694 16700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:17.663535 16715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.663707 16700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:15:17.663805 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "9ca95638f0194c53aef44fc1c1d3b1dc"
format_stamp: "Formatted at 2026-09-25 05:15:17 on dist-test-slave-9tgz"
server_key: "83f1c9c40dfc25e67d127eb049142ca8"
server_key_iv: "e2dc1bb88ea94ecadd07c0b077cfe7ae"
server_key_version: "encryptionkey@0"
I20260925 05:15:17.663885 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:17.677340 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:17.679553 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:17.679731 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:17.679975 16700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:17.680384 16700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:17.680433 16700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.680476 16700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:17.680503 16700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:17.686487 16700 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:35587
I20260925 05:15:17.686528 16828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:35587 every 8 connection(s)
I20260925 05:15:17.686951 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313317, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:17.696244 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.687730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:43817 (local address 127.13.160.190:45477)
0925 05:15:17.687787 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:17.687789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:17.687998 (+ 209us) server_negotiation.cc:407] Connection header received
0925 05:15:17.688490 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:17.688490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:17.688500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:17.688566 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:17.689574 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.690125 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.691163 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.691370 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.691426 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.693711 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:17.693713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:17.693713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:17.693720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:17.695062 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.695447 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.695448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.695448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.695504 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:17.695713 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:17.695714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:17.695714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:17.695822 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:17.695871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:17.696111 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 05:15:17.696175 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 05:15:17.696484 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:17.687688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45477 (local address 127.13.160.131:43817)
0925 05:15:17.687867 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:17.687911 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:17.688403 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:17.688643 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:17.688646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:17.689272 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:17.689483 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.689489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.690214 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.690215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.691077 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:17.691079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:17.691453 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:17.691454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:17.691625 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:17.692151 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:17.692160 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:17.693602 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:17.695137 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.695143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.695146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.695379 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.695559 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:17.695559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:17.695560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:17.695663 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:17.695872 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:17.695875 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:17.696002 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 05:15:17.696086 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 05:15:17.696219 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260925 05:15:17.696997 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16700
I20260925 05:15:17.697011 16829 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45477
I20260925 05:15:17.697091 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:17.697110 16829 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:17.697268 13954 external_mini_cluster.cc:1428] Setting key a9dbe3ee27d60fcc5738549a633e0682
I20260925 05:15:17.697327 16829 heartbeater.cc:511] Master 127.13.160.190:45477 requested a full tablet report, sending...
I20260925 05:15:17.697815 16370 ts_manager.cc:194] Registered new tserver with Master: 9ca95638f0194c53aef44fc1c1d3b1dc (127.13.160.131:35587)
I20260925 05:15:17.698184 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:17.698304 16370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:43817
W20260925 05:15:17.709821 16427 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:17.708756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35256 (local address 127.13.160.190:45477)
0925 05:15:17.708840 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:17.708843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:17.708942 (+ 99us) server_negotiation.cc:407] Connection header received
0925 05:15:17.709036 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:17.709037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:17.709049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:17.709117 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:17.709732 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35256: BlockingRecv error: recv got EOF from 127.0.0.1:35256 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:15:17.710698 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16432
I20260925 05:15:17.715438 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16566
I20260925 05:15:17.719893 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16700
I20260925 05:15:17.724298 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16338
2026-09-25T05:15:17Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1854 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16841](info): set up 2 sockets
Sep 25 05:15:17 dist-test-slave-9tgz krb5kdc[16841](info): commencing operation
krb5kdc: starting...
W20260925 05:15:18.759637 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:18 dist-test-slave-9tgz krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:18.880240 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37751
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:18.963927 16854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:18.964206 16854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:18.964249 16854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:18.965505 16854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:18.965579 16854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:18.965601 16854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:18.965639 16854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:18.965669 16854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:18.967404 16854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:18.967717 16854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:18.967962 16854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:18.972976 16859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:18.972976 16860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:18.973117 16854 server_base.cc:1034] running on GCE node
W20260925 05:15:18.972970 16862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:18.973485 16854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:18.973804 16854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:18.974944 16854 hybrid_clock.cc:648] HybridClock initialized: now 1790313318974927 us; error 30 us; skew 500 ppm
Sep 25 05:15:18 dist-test-slave-9tgz krb5kdc[16841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313318, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:18.982745 16854 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:18.983278 16854 webserver.cc:460] Webserver started at http://127.13.160.190:37989/ using document root <none> and password file <none>
I20260925 05:15:18.983534 16854 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:18.983584 16854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:18.983750 16854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:18.984954 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "1563d47a331f4d31b34114491f9c1a4c"
format_stamp: "Formatted at 2026-09-25 05:15:18 on dist-test-slave-9tgz"
server_key: "4be1d996e496c05f2f3c99fde008dbe0"
server_key_iv: "237cc84b19d2cb0fa003b4b929833301"
server_key_version: "encryptionkey@0"
I20260925 05:15:18.985356 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "1563d47a331f4d31b34114491f9c1a4c"
format_stamp: "Formatted at 2026-09-25 05:15:18 on dist-test-slave-9tgz"
server_key: "4be1d996e496c05f2f3c99fde008dbe0"
server_key_iv: "237cc84b19d2cb0fa003b4b929833301"
server_key_version: "encryptionkey@0"
I20260925 05:15:18.986804 16854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:18.987695 16869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:18.987901 16854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:18.988003 16854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "1563d47a331f4d31b34114491f9c1a4c"
format_stamp: "Formatted at 2026-09-25 05:15:18 on dist-test-slave-9tgz"
server_key: "4be1d996e496c05f2f3c99fde008dbe0"
server_key_iv: "237cc84b19d2cb0fa003b4b929833301"
server_key_version: "encryptionkey@0"
I20260925 05:15:18.988085 16854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:19.008178 16854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:19.010262 16854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:19.010425 16854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:19.014707 16854 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:37751
I20260925 05:15:19.014743 16921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:37751 every 8 connection(s)
I20260925 05:15:19.015348 16854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:19.015480 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16854
I20260925 05:15:19.015574 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:19.015771 13954 external_mini_cluster.cc:1428] Setting key 61cbf3bccebcea750516b3d7ca22f1ca
I20260925 05:15:19.016203 16922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:19.021741 16922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c: Bootstrap starting.
I20260925 05:15:19.022713 16922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:19.023103 16922 log.cc:824] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c: Log is configured to *not* fsync() on all Append() calls
Sep 25 05:15:19 dist-test-slave-9tgz krb5kdc[16841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:19.024808 16922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c: No bootstrap required, opened a new log
I20260925 05:15:19.027398 16922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } }
I20260925 05:15:19.027601 16922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:19.027644 16922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1563d47a331f4d31b34114491f9c1a4c, State: Initialized, Role: FOLLOWER
I20260925 05:15:19.027809 16922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } }
I20260925 05:15:19.027889 16922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:19.027930 16922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:19.027980 16922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:19.028827 16922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } }
I20260925 05:15:19.028955 16922 leader_election.cc:302] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1563d47a331f4d31b34114491f9c1a4c; no voters:
I20260925 05:15:19.029157 16922 leader_election.cc:288] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:19.029225 16927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:19.029383 16927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [term 1 LEADER]: Becoming Leader. State: Replica: 1563d47a331f4d31b34114491f9c1a4c, State: Running, Role: LEADER
I20260925 05:15:19.029453 16922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:19.029484 16927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } }
I20260925 05:15:19.029860 16928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1563d47a331f4d31b34114491f9c1a4c. Latest consensus state: current_term: 1 leader_uuid: "1563d47a331f4d31b34114491f9c1a4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } } }
I20260925 05:15:19.029939 16928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:19.030193 16934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:19.030707 16934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:19.031404 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:19.016323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.13.160.190:37751)
0925 05:15:19.016540 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:19.016551 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:19.016630 (+ 79us) server_negotiation.cc:407] Connection header received
0925 05:15:19.017206 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:19.017213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:19.017266 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:19.017418 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:19.018239 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:19.019677 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:19.020923 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:19.021717 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:19.021850 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:19.024994 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:19.025000 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:19.025002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:19.025017 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:19.029210 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:19.029755 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:19.029759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:19.029759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:19.029881 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:19.030235 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:19.030237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:19.030237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:19.030620 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:19.030799 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:19.031093 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 05:15:19.031249 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260925 05:15:19.032166 16944 mvcc.cc:205] Tried to move back new op lower bound from 7333123354753654784 to 7333123354745311232. Current Snapshot: MvccSnapshot[applied={T|T < 7333123354753654784}]
I20260925 05:15:19.032176 16929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1563d47a331f4d31b34114491f9c1a4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1563d47a331f4d31b34114491f9c1a4c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37751 } } }
I20260925 05:15:19.032395 16929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:19.032621 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:15:19.032735 16934 catalog_manager.cc:1347] Generated new cluster ID: e9fe1b193f90477ab9ab4417cc8bbc97
I20260925 05:15:19.032801 16934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:19.050033 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:19.045444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.13.160.190:37751)
0925 05:15:19.045492 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:19.045495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:19.045576 (+ 81us) server_negotiation.cc:407] Connection header received
0925 05:15:19.045646 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:19.045647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:19.045658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:19.045737 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:19.046162 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:19.046986 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:19.048052 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:19.048503 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:19.048596 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:19.049429 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:19.049431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:19.049431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:19.049694 (+ 263us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:19.049715 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:19.049722 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:19.049785 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:19.049884 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 05:15:19.049912 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:15:19.050518 16934 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:19.051527 16934 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:19.060877 16934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1563d47a331f4d31b34114491f9c1a4c: Generated new TSK 0
I20260925 05:15:19.061121 16934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:19.070399 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16854
2026-09-25T05:15:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1344 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:19 dist-test-slave-9tgz krb5kdc[16955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:19 dist-test-slave-9tgz krb5kdc[16955](info): set up 2 sockets
Sep 25 05:15:19 dist-test-slave-9tgz krb5kdc[16955](info): commencing operation
krb5kdc: starting...
W20260925 05:15:20.103631 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:20.221833 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41079
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:20.305938 16968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:20.306209 16968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:20.306252 16968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:20.307574 16968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:20.307657 16968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:20.307680 16968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:20.307703 16968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:20.307731 16968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:20.309302 16968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:20.309619 16968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:20.309856 16968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:20.315064 16976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.314986 16974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:20.315006 16968 server_base.cc:1034] running on GCE node
W20260925 05:15:20.314982 16973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:20.315501 16968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:20.315793 16968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:20.316982 16968 hybrid_clock.cc:648] HybridClock initialized: now 1790313320316967 us; error 46 us; skew 500 ppm
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:20.319698 16968 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:20.320196 16968 webserver.cc:460] Webserver started at http://127.13.160.190:44249/ using document root <none> and password file <none>
I20260925 05:15:20.320423 16968 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:20.320492 16968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:20.320642 16968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:20.321740 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "a67aa4f909444333a84422d97d1b6112"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "fab5a037d68383d9d28f61537f369c44"
server_key_iv: "6a7b710c541fc7e8399c73e6126835d9"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.322119 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "a67aa4f909444333a84422d97d1b6112"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "fab5a037d68383d9d28f61537f369c44"
server_key_iv: "6a7b710c541fc7e8399c73e6126835d9"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.323581 16968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:20.324357 16983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.324568 16968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:20.324687 16968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "a67aa4f909444333a84422d97d1b6112"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "fab5a037d68383d9d28f61537f369c44"
server_key_iv: "6a7b710c541fc7e8399c73e6126835d9"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.324767 16968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:20.338629 16968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:20.340647 16968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:20.340811 16968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:20.344866 16968 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:41079
I20260925 05:15:20.344928 17035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:41079 every 8 connection(s)
I20260925 05:15:20.345315 16968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:20.346319 17036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:20.346645 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 16968
I20260925 05:15:20.346768 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:20.346998 13954 external_mini_cluster.cc:1428] Setting key d09f8a1dfca9a9f3f8a54b79551cb66e
I20260925 05:15:20.349870 17036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112: Bootstrap starting.
I20260925 05:15:20.350605 17036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:20.350956 17036 log.cc:824] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:20.351737 17036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112: No bootstrap required, opened a new log
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:20.354043 17036 raft_consensus.cc:348] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } }
I20260925 05:15:20.354233 17036 raft_consensus.cc:374] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:20.354269 17036 raft_consensus.cc:729] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a67aa4f909444333a84422d97d1b6112, State: Initialized, Role: FOLLOWER
I20260925 05:15:20.354424 17036 consensus_queue.cc:260] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } }
I20260925 05:15:20.354535 17036 raft_consensus.cc:388] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:20.354575 17036 raft_consensus.cc:482] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:20.354647 17036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:20.355470 17036 raft_consensus.cc:504] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } }
I20260925 05:15:20.355598 17036 leader_election.cc:302] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a67aa4f909444333a84422d97d1b6112; no voters:
I20260925 05:15:20.355791 17036 leader_election.cc:288] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:20.355849 17041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:20.356015 17041 raft_consensus.cc:686] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [term 1 LEADER]: Becoming Leader. State: Replica: a67aa4f909444333a84422d97d1b6112, State: Running, Role: LEADER
I20260925 05:15:20.356102 17036 sys_catalog.cc:564] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:20.356117 17041 consensus_queue.cc:237] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } }
I20260925 05:15:20.356488 17041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a67aa4f909444333a84422d97d1b6112. Latest consensus state: current_term: 1 leader_uuid: "a67aa4f909444333a84422d97d1b6112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } } }
I20260925 05:15:20.356570 17041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:20.356727 17043 sys_catalog.cc:455] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a67aa4f909444333a84422d97d1b6112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a67aa4f909444333a84422d97d1b6112" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41079 } } }
I20260925 05:15:20.356791 17043 sys_catalog.cc:458] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:20.356907 17045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:20.357476 17045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:20.358942 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.347540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.13.160.190:41079)
0925 05:15:20.347725 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:20.347734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:20.347761 (+ 27us) server_negotiation.cc:407] Connection header received
0925 05:15:20.348210 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:20.348218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:20.348274 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:20.348395 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:20.349165 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.350122 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.351269 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.351765 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.351916 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.354660 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:20.354665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:20.354666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:20.354680 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:20.356896 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.357366 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.357369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.357369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.357477 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.357807 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.357808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.357808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.358143 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:20.358318 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:20.358437 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 05:15:20.358711 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260925 05:15:20.359340 17045 catalog_manager.cc:1347] Generated new cluster ID: f510d9c5a8e944debd3e129fd297739c
I20260925 05:15:20.359392 17045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:20.375916 17045 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:20.376457 17045 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:20.386234 17045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a67aa4f909444333a84422d97d1b6112: Generated new TSK 0
I20260925 05:15:20.386507 17045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:20.422518 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:20.506742 17062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:20.507071 17062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:20.507117 17062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:20.508549 17062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:20.508647 17062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:20.508701 17062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:20.510390 17062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:20.510722 17062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:20.511011 17062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:20.516206 17068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.516202 17070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.516196 17067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:20.516546 17062 server_base.cc:1034] running on GCE node
I20260925 05:15:20.516762 17062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:20.516983 17062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:20.518133 17062 hybrid_clock.cc:648] HybridClock initialized: now 1790313320518122 us; error 34 us; skew 500 ppm
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:20.520773 17062 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:20.521257 17062 webserver.cc:460] Webserver started at http://127.13.160.129:34607/ using document root <none> and password file <none>
I20260925 05:15:20.521459 17062 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:20.521502 17062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:20.521664 17062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:20.522742 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "16ebc40e8652488bae253b1db3fee9b4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "cca58a211ff1dd714bb30cc0e8a7ca1e"
server_key_iv: "aae498edbaa77d841c29d1decb349732"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.523128 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "16ebc40e8652488bae253b1db3fee9b4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "cca58a211ff1dd714bb30cc0e8a7ca1e"
server_key_iv: "aae498edbaa77d841c29d1decb349732"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.524468 17062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:20.525179 17077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.525350 17062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:20.525426 17062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "16ebc40e8652488bae253b1db3fee9b4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "cca58a211ff1dd714bb30cc0e8a7ca1e"
server_key_iv: "aae498edbaa77d841c29d1decb349732"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.525483 17062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:20.544612 17062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:20.546648 17062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:20.546772 17062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:20.546988 17062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:20.547326 17062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:20.547360 17062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.547416 17062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:20.547443 17062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.553565 17062 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:45945
I20260925 05:15:20.553612 17190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:45945 every 8 connection(s)
I20260925 05:15:20.553961 17062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:20.558282 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17062
I20260925 05:15:20.558394 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:20.558581 13954 external_mini_cluster.cc:1428] Setting key e68fa00b35dbf75b619926eac28de034
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:20.564049 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.554708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:37541 (local address 127.13.160.190:41079)
0925 05:15:20.554774 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:20.554775 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:20.555017 (+ 242us) server_negotiation.cc:407] Connection header received
0925 05:15:20.555506 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:20.555507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:20.555517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:20.555568 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:20.556654 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.557264 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.558280 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.558618 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.558704 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.561017 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:20.561019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:20.561019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:20.561027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:20.562511 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.563007 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.563008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.563009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.563070 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.563357 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.563358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.563359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.563484 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:20.563553 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:20.563860 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 05:15:20.563955 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:15:20.564258 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.554695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41079 (local address 127.13.160.129:37541)
0925 05:15:20.554846 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:20.554919 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:20.555420 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:20.555646 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:20.555650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:20.556314 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:20.556543 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.556548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.557345 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.557347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.558194 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.558196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.558743 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.558744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.558925 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.559462 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:20.559474 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:20.560919 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:20.562624 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.562632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.562635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.562933 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.563152 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.563153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.563154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.563276 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.563562 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:20.563569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:20.563725 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 05:15:20.563826 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 05:15:20.563998 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260925 05:15:20.564810 17191 heartbeater.cc:347] Connected to a master server at 127.13.160.190:41079
I20260925 05:15:20.564944 17191 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:20.565155 17191 heartbeater.cc:511] Master 127.13.160.190:41079 requested a full tablet report, sending...
I20260925 05:15:20.565718 17000 ts_manager.cc:194] Registered new tserver with Master: 16ebc40e8652488bae253b1db3fee9b4 (127.13.160.129:45945)
I20260925 05:15:20.566341 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:37541
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:20.585415 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:20.668861 17196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:20.669103 17196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:20.669138 17196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:20.670395 17196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:20.670444 17196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:20.670472 17196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:20.671994 17196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:20.672410 17196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:20.672673 17196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:20.677840 17202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.677865 17204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.677850 17201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:20.678072 17196 server_base.cc:1034] running on GCE node
I20260925 05:15:20.678342 17196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:20.678646 17196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:20.679833 17196 hybrid_clock.cc:648] HybridClock initialized: now 1790313320679799 us; error 54 us; skew 500 ppm
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:20.682472 17196 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:20.682952 17196 webserver.cc:460] Webserver started at http://127.13.160.130:35135/ using document root <none> and password file <none>
I20260925 05:15:20.683157 17196 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:20.683202 17196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:20.683297 17196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:20.684605 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "35a46b7b702940aa8a8a1041d768aac4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "1252e09a11afc43523f1877470934735"
server_key_iv: "6aed977e106ad8491f0aaa81b1845c19"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.684950 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "35a46b7b702940aa8a8a1041d768aac4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "1252e09a11afc43523f1877470934735"
server_key_iv: "6aed977e106ad8491f0aaa81b1845c19"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.686337 17196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:20.687112 17211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.687270 17196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:20.687361 17196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "35a46b7b702940aa8a8a1041d768aac4"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "1252e09a11afc43523f1877470934735"
server_key_iv: "6aed977e106ad8491f0aaa81b1845c19"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.687423 17196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:20.701320 17196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:20.703533 17196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:20.703667 17196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:20.703863 17196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:20.704244 17196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:20.704301 17196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.704327 17196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:20.704340 17196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.710141 17196 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38217
I20260925 05:15:20.710222 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38217 every 8 connection(s)
I20260925 05:15:20.710628 17196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:20.719990 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.711566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:36887 (local address 127.13.160.190:41079)
0925 05:15:20.711638 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:20.711641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:20.711841 (+ 200us) server_negotiation.cc:407] Connection header received
0925 05:15:20.712340 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:20.712341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:20.712352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:20.712404 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:20.713452 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.713941 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.715002 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.715184 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.715238 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.717467 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:20.717480 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:20.717480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:20.717487 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:20.718713 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.719170 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.719171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.719171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.719224 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.719432 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.719433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.719433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.719541 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:20.719590 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:20.719853 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 05:15:20.719917 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:20.720178 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.711505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41079 (local address 127.13.160.130:36887)
0925 05:15:20.711711 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:20.711752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:20.712254 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:20.712509 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:20.712514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:20.713157 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:20.713366 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.713371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.714031 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.714032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.714866 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.714868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.715257 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.715258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.715370 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.715898 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:20.715909 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:20.717370 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:20.718789 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.718795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.718797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.719099 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.719280 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.719281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.719281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.719385 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.719596 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:20.719602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:20.719741 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 05:15:20.719829 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 05:15:20.719958 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260925 05:15:20.720626 17325 heartbeater.cc:347] Connected to a master server at 127.13.160.190:41079
I20260925 05:15:20.720744 17325 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:20.720804 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17196
I20260925 05:15:20.720907 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:20.720947 17325 heartbeater.cc:511] Master 127.13.160.190:41079 requested a full tablet report, sending...
I20260925 05:15:20.721096 13954 external_mini_cluster.cc:1428] Setting key 3878cab03b85ee1f09dbad5e5ab96d1f
I20260925 05:15:20.721366 17000 ts_manager.cc:194] Registered new tserver with Master: 35a46b7b702940aa8a8a1041d768aac4 (127.13.160.130:38217)
I20260925 05:15:20.721997 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:36887
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:20.746037 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:20.829290 17330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:20.829551 17330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:20.829594 17330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:20.830828 17330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:20.830895 17330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:20.830973 17330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:20.832518 17330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:41079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:20.832815 17330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:20.833056 17330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:20.838229 17336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.838253 17338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:20.838326 17335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:20.838573 17330 server_base.cc:1034] running on GCE node
I20260925 05:15:20.838851 17330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:20.839190 17330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:20.840349 17330 hybrid_clock.cc:648] HybridClock initialized: now 1790313320840332 us; error 35 us; skew 500 ppm
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:20.842988 17330 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:20.843415 17330 webserver.cc:460] Webserver started at http://127.13.160.131:33705/ using document root <none> and password file <none>
I20260925 05:15:20.843609 17330 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:20.843652 17330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:20.843745 17330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:20.844837 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "3535634ab8da4ef8a6ef5801ce4f5d28"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "95d8c1b225a493939bc556c17f0d6cc2"
server_key_iv: "e1284674e4aa9c106894ae3de19b2aba"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.845170 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "3535634ab8da4ef8a6ef5801ce4f5d28"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "95d8c1b225a493939bc556c17f0d6cc2"
server_key_iv: "e1284674e4aa9c106894ae3de19b2aba"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.846537 17330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:20.847371 17345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.847590 17330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:15:20.847675 17330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "3535634ab8da4ef8a6ef5801ce4f5d28"
format_stamp: "Formatted at 2026-09-25 05:15:20 on dist-test-slave-9tgz"
server_key: "95d8c1b225a493939bc556c17f0d6cc2"
server_key_iv: "e1284674e4aa9c106894ae3de19b2aba"
server_key_version: "encryptionkey@0"
I20260925 05:15:20.847738 17330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:20.873310 17330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:20.875417 17330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:20.875538 17330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:20.875747 17330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:20.876088 17330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:20.876123 17330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.876186 17330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:20.876215 17330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:20.882182 17330 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:37893
I20260925 05:15:20.882217 17458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:37893 every 8 connection(s)
I20260925 05:15:20.882582 17330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:20.891480 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17330
I20260925 05:15:20.891595 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:20.891783 13954 external_mini_cluster.cc:1428] Setting key bff2eb980f8eb9b9b1ef7ceb552746e8
I20260925 05:15:20.892346 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.883387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:42227 (local address 127.13.160.190:41079)
0925 05:15:20.883466 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:20.883469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:20.883642 (+ 173us) server_negotiation.cc:407] Connection header received
0925 05:15:20.884130 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:20.884131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:20.884142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:20.884200 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:20.885143 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.885648 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.886918 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.887125 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.887212 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.889393 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:20.889395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:20.889395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:20.889403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:20.890711 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.891180 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.891182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.891182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.891241 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.891526 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.891527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.891528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.891712 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:20.891774 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:20.892133 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 05:15:20.892246 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 05:15:20.892553 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.883352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41079 (local address 127.13.160.131:42227)
0925 05:15:20.883521 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:20.883558 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:20.884045 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:20.884263 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:20.884266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:20.884855 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:20.885056 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.885062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.885728 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.885728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.886799 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:20.886802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.887228 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.887229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:20.887358 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.887837 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:20.887846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:20.889285 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:20.890793 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.890801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.890804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.891099 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.891330 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:20.891330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:20.891331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:20.891464 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:20.891783 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:20.891790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:20.891955 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 05:15:20.892070 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:15:20.892280 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260925 05:15:20.892962 17459 heartbeater.cc:347] Connected to a master server at 127.13.160.190:41079
I20260925 05:15:20.893047 17459 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:20.893208 17459 heartbeater.cc:511] Master 127.13.160.190:41079 requested a full tablet report, sending...
I20260925 05:15:20.893553 17000 ts_manager.cc:194] Registered new tserver with Master: 3535634ab8da4ef8a6ef5801ce4f5d28 (127.13.160.131:37893)
I20260925 05:15:20.893896 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:42227
I20260925 05:15:20.903627 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:20.926692 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:20.919547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.13.160.190:41079)
0925 05:15:20.919634 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:20.919637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:20.919747 (+ 110us) server_negotiation.cc:407] Connection header received
0925 05:15:20.919842 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:20.919843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:20.919854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:20.919925 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:20.920523 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.921024 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.921771 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:20.921965 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:20.922037 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:20.924314 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:20.924316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:20.924316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:20.924325 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:20.925526 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.925868 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.925869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.925870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.925926 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:20.926114 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:20.926115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:20.926115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:20.926276 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:20.926379 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:20.926485 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 05:15:20.926569 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 05:15:20.926991 17000 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50366
I20260925 05:15:20.927409 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17062
I20260925 05:15:20.932117 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17196
I20260925 05:15:20.936357 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17330
I20260925 05:15:20.940523 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 16968
2026-09-25T05:15:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1872 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[17471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[17471](info): set up 2 sockets
Sep 25 05:15:20 dist-test-slave-9tgz krb5kdc[17471](info): commencing operation
krb5kdc: starting...
W20260925 05:15:21.971781 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:22.105731 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40739
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:22.191201 17484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:22.191437 17484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:22.191462 17484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:22.192752 17484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:22.192807 17484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:22.192821 17484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:22.192834 17484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:22.192888 17484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:22.194545 17484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:22.194844 17484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:22.195123 17484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:22.200235 17484 server_base.cc:1034] running on GCE node
W20260925 05:15:22.200160 17490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.200237 17492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.200161 17489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.200661 17484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:22.201004 17484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:22.202185 17484 hybrid_clock.cc:648] HybridClock initialized: now 1790313322202171 us; error 47 us; skew 500 ppm
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:22.204808 17484 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:22.205255 17484 webserver.cc:460] Webserver started at http://127.13.160.190:42171/ using document root <none> and password file <none>
I20260925 05:15:22.205457 17484 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:22.205499 17484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:22.205678 17484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:22.206791 17484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "8fc430f4fe7e4f5e829740f93bcde29c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "0f80494d099b95742ba5262841e9c199"
server_key_iv: "fa2cc00be7b8401e0a1114178b68db9d"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.207198 17484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "8fc430f4fe7e4f5e829740f93bcde29c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "0f80494d099b95742ba5262841e9c199"
server_key_iv: "fa2cc00be7b8401e0a1114178b68db9d"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.208555 17484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:22.209307 17499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.209508 17484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:22.209601 17484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "8fc430f4fe7e4f5e829740f93bcde29c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "0f80494d099b95742ba5262841e9c199"
server_key_iv: "fa2cc00be7b8401e0a1114178b68db9d"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.209684 17484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:22.224503 17484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:22.226517 17484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:22.226686 17484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:22.230768 17484 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40739
I20260925 05:15:22.230836 17551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40739 every 8 connection(s)
I20260925 05:15:22.231410 17484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:22.232163 17552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:22.234555 17552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c: Bootstrap starting.
I20260925 05:15:22.235255 17552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:22.235514 17552 log.cc:824] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:22.236227 17552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c: No bootstrap required, opened a new log
I20260925 05:15:22.237865 17552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } }
I20260925 05:15:22.238034 17552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:22.238057 17552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc430f4fe7e4f5e829740f93bcde29c, State: Initialized, Role: FOLLOWER
I20260925 05:15:22.238175 17552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } }
I20260925 05:15:22.238266 17552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:22.238301 17552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:22.238355 17552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:22.239403 17552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } }
I20260925 05:15:22.239548 17552 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fc430f4fe7e4f5e829740f93bcde29c; no voters:
I20260925 05:15:22.239768 17552 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:22.239822 17555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:22.240033 17555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [term 1 LEADER]: Becoming Leader. State: Replica: 8fc430f4fe7e4f5e829740f93bcde29c, State: Running, Role: LEADER
I20260925 05:15:22.240074 17552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:22.240139 17555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } }
I20260925 05:15:22.240525 17557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc430f4fe7e4f5e829740f93bcde29c. Latest consensus state: current_term: 1 leader_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } } }
I20260925 05:15:22.240631 17557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:22.240522 17556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc430f4fe7e4f5e829740f93bcde29c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40739 } } }
I20260925 05:15:22.240777 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17484
I20260925 05:15:22.240797 17556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:22.240883 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:22.240967 17560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:22.241088 13954 external_mini_cluster.cc:1428] Setting key 25aa636723b1bf5e018f0c026bc3ebb3
I20260925 05:15:22.241564 17560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:22.243288 17560 catalog_manager.cc:1347] Generated new cluster ID: de1555d38966460fb9c20aa160b05b33
I20260925 05:15:22.243348 17560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:22.252096 17560 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:22.252787 17560 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:22.257606 17560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fc430f4fe7e4f5e829740f93bcde29c: Generated new TSK 0
I20260925 05:15:22.257807 17560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:22.259382 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.241758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.13.160.190:40739)
0925 05:15:22.247444 (+ 5686us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.247456 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.247500 (+ 44us) server_negotiation.cc:407] Connection header received
0925 05:15:22.247639 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.247646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.247706 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.247873 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.248776 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.250312 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.251529 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.252202 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.252314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.255318 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.255324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.255324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.255339 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.257630 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.258113 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.258117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.258117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.258237 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.258447 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.258448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.258448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.258666 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.258785 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.258985 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.259172 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5624,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:22.289620 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:22.375708 17578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:22.375938 17578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:22.375967 17578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:22.377257 17578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:22.377315 17578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:22.377345 17578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:22.379101 17578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:22.379580 17578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:22.379855 17578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:22.385010 17583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.385013 17584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.385010 17586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.385321 17578 server_base.cc:1034] running on GCE node
I20260925 05:15:22.385538 17578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:22.385800 17578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:22.386955 17578 hybrid_clock.cc:648] HybridClock initialized: now 1790313322386934 us; error 33 us; skew 500 ppm
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:22.389596 17578 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:22.390022 17578 webserver.cc:460] Webserver started at http://127.13.160.129:38001/ using document root <none> and password file <none>
I20260925 05:15:22.390226 17578 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:22.390268 17578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:22.390362 17578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:22.391495 17578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "7bf995e0ad214c3486658927b57eda39"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "d8a574b2d7dc5f706c01f52b3435325b"
server_key_iv: "5e1bcbbdb23d3f219b53aaea4115824e"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.391820 17578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "7bf995e0ad214c3486658927b57eda39"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "d8a574b2d7dc5f706c01f52b3435325b"
server_key_iv: "5e1bcbbdb23d3f219b53aaea4115824e"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.393138 17578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:22.393848 17593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.394003 17578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:22.394091 17578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "7bf995e0ad214c3486658927b57eda39"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "d8a574b2d7dc5f706c01f52b3435325b"
server_key_iv: "5e1bcbbdb23d3f219b53aaea4115824e"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.394160 17578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:22.423892 17578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:22.425997 17578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:22.426129 17578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:22.426375 17578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:22.426723 17578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:22.426762 17578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.426786 17578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:22.426836 17578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.432631 17578 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:33505
I20260925 05:15:22.432717 17706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:33505 every 8 connection(s)
I20260925 05:15:22.433039 17578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:22.435014 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17578
I20260925 05:15:22.435122 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:22.435295 13954 external_mini_cluster.cc:1428] Setting key f28f5e98fdf6755a462bdf011e1f1871
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
I20260925 05:15:22.444389 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.433807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:58187 (local address 127.13.160.190:40739)
0925 05:15:22.433875 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.433877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.434110 (+ 233us) server_negotiation.cc:407] Connection header received
0925 05:15:22.435234 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.435235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.435251 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.435306 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.436150 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.436702 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.438341 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.438546 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.438609 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.441277 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.441278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.441278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.441287 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.442854 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.443340 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.443342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.443342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.443405 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.443661 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.443662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.443663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.443794 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.443869 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.444184 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.444281 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:15:22.444583 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.433785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40739 (local address 127.13.160.129:58187)
0925 05:15:22.433970 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:22.434015 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:22.435078 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:22.435386 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:22.435390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:22.435760 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:22.436059 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.436065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.436825 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.436826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.438245 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.438248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.438641 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.438642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.438801 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.439374 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:22.439385 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:22.441176 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:22.442988 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.442996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.442999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.443251 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.443469 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.443470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.443471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.443591 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.443858 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:22.443865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:22.444023 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 05:15:22.444158 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 05:15:22.444335 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260925 05:15:22.445151 17707 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40739
I20260925 05:15:22.445268 17707 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:22.445428 17707 heartbeater.cc:511] Master 127.13.160.190:40739 requested a full tablet report, sending...
I20260925 05:15:22.445984 17516 ts_manager.cc:194] Registered new tserver with Master: 7bf995e0ad214c3486658927b57eda39 (127.13.160.129:33505)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
I20260925 05:15:22.446767 17516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:58187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:22.461540 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:22.544751 17712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:22.544986 17712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:22.545013 17712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:22.546181 17712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:22.546222 17712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:22.546249 17712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:22.547822 17712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:22.548033 17712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:22.548228 17712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:22.553329 17720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.553380 17717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.553400 17712 server_base.cc:1034] running on GCE node
W20260925 05:15:22.553380 17718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.553785 17712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:22.554090 17712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:22.555249 17712 hybrid_clock.cc:648] HybridClock initialized: now 1790313322555228 us; error 37 us; skew 500 ppm
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:22.557979 17712 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:22.558421 17712 webserver.cc:460] Webserver started at http://127.13.160.130:37885/ using document root <none> and password file <none>
I20260925 05:15:22.558638 17712 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:22.558691 17712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:22.558786 17712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:22.559918 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "48ac95fbc0934dd9840352729e98664c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "5aa41e8df817cf8afe21cf004278fd49"
server_key_iv: "53dd52fadefcdcad5094d15e7b6273a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.560253 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "48ac95fbc0934dd9840352729e98664c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "5aa41e8df817cf8afe21cf004278fd49"
server_key_iv: "53dd52fadefcdcad5094d15e7b6273a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.561626 17712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:22.562336 17727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.562513 17712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:22.562606 17712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "48ac95fbc0934dd9840352729e98664c"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "5aa41e8df817cf8afe21cf004278fd49"
server_key_iv: "53dd52fadefcdcad5094d15e7b6273a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.562682 17712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:22.574687 17712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:22.577207 17712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:22.577374 17712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:22.577628 17712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:22.577960 17712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:22.578001 17712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.578059 17712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:22.578088 17712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.584071 17712 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38087
I20260925 05:15:22.584160 17840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38087 every 8 connection(s)
I20260925 05:15:22.584482 17712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:22.586536 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17712
I20260925 05:15:22.586663 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:22.586858 13954 external_mini_cluster.cc:1428] Setting key 708e34a7d23de5a0d40be52a6852d763
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:22.595073 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.585269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:33229 (local address 127.13.160.190:40739)
0925 05:15:22.585327 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.585329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.585540 (+ 211us) server_negotiation.cc:407] Connection header received
0925 05:15:22.586510 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.586512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.586533 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.586617 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.587414 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.587929 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.589301 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.589495 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.589607 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.592170 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.592172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.592172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.592180 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.593641 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.594065 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.594067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.594067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.594127 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.594375 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.594376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.594376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.594513 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.594588 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.594862 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.594972 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
I20260925 05:15:22.595286 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.585262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40739 (local address 127.13.160.130:33229)
0925 05:15:22.585412 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:22.585449 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:22.586397 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:22.586721 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:22.586726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:22.587103 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:22.587318 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.587323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.588027 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.588028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.589198 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.589201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.589617 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.589618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.589768 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.590268 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:22.590279 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:22.592062 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:22.593730 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.593739 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.593741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.593989 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.594192 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.594193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.594193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.594308 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.594578 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:22.594585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:22.594734 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 05:15:22.594843 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 05:15:22.595039 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":91,"threads_started":1}
I20260925 05:15:22.595815 17841 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40739
I20260925 05:15:22.595927 17841 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:22.596128 17841 heartbeater.cc:511] Master 127.13.160.190:40739 requested a full tablet report, sending...
I20260925 05:15:22.596508 17516 ts_manager.cc:194] Registered new tserver with Master: 48ac95fbc0934dd9840352729e98664c (127.13.160.130:38087)
I20260925 05:15:22.596957 17516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:33229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:22.613554 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:22.695967 17846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:22.696249 17846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:22.696290 17846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:22.697609 17846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:22.697674 17846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:22.697724 17846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:22.699421 17846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:22.699723 17846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:22.699965 17846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:22.705161 17854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.705199 17846 server_base.cc:1034] running on GCE node
W20260925 05:15:22.705150 17852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:22.705149 17851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:22.705593 17846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:22.705875 17846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:22.707037 17846 hybrid_clock.cc:648] HybridClock initialized: now 1790313322707030 us; error 38 us; skew 500 ppm
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:22.709717 17846 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:22.710175 17846 webserver.cc:460] Webserver started at http://127.13.160.131:41715/ using document root <none> and password file <none>
I20260925 05:15:22.710372 17846 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:22.710415 17846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:22.710511 17846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:22.711643 17846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "a8f55e88053441f48a2a3c4b65e2ca7e"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "ec5370fde1b548efeb114f2b883c19df"
server_key_iv: "c414c2b9ce00388622fcfe2652c11562"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.712008 17846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "a8f55e88053441f48a2a3c4b65e2ca7e"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "ec5370fde1b548efeb114f2b883c19df"
server_key_iv: "c414c2b9ce00388622fcfe2652c11562"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.713495 17846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:15:22.714205 17861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.714373 17846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:22.714460 17846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "a8f55e88053441f48a2a3c4b65e2ca7e"
format_stamp: "Formatted at 2026-09-25 05:15:22 on dist-test-slave-9tgz"
server_key: "ec5370fde1b548efeb114f2b883c19df"
server_key_iv: "c414c2b9ce00388622fcfe2652c11562"
server_key_version: "encryptionkey@0"
I20260925 05:15:22.714520 17846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:22.722349 17846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:22.724774 17846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:22.724897 17846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:22.725090 17846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:22.725441 17846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:22.725478 17846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.725522 17846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:22.725548 17846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:22.731501 17846 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:40725
I20260925 05:15:22.731585 17974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:40725 every 8 connection(s)
I20260925 05:15:22.731916 17846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:22.738731 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 17846
I20260925 05:15:22.738857 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:22.739104 13954 external_mini_cluster.cc:1428] Setting key c6795ad7cb9f62c5c13b6501a21633f5
I20260925 05:15:22.742306 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.732742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:38851 (local address 127.13.160.190:40739)
0925 05:15:22.732803 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.732805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.733016 (+ 211us) server_negotiation.cc:407] Connection header received
0925 05:15:22.734019 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.734020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.734034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.734095 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.734772 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.735354 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.736408 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.736623 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.736714 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.738990 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.738992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.738992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.739001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.740786 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.741258 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.741259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.741260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.741350 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.741609 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.741610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.741610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.741748 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.741820 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.742118 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.742212 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 05:15:22.742527 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.732720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40739 (local address 127.13.160.131:38851)
0925 05:15:22.732890 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:22.732931 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:22.733897 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:22.734184 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:22.734187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:22.734471 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:22.734675 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.734678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.735468 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.735469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.736305 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:22.736307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.736726 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.736727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:22.736858 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.737364 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:22.737372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:22.738821 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:22.740863 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.740870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.740872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.741164 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.741423 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:22.741423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:22.741424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:22.741541 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:22.741820 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:22.741828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:22.741972 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 05:15:22.742083 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 05:15:22.742267 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260925 05:15:22.743024 17975 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40739
I20260925 05:15:22.743145 17975 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:22.743338 17975 heartbeater.cc:511] Master 127.13.160.190:40739 requested a full tablet report, sending...
I20260925 05:15:22.743758 17516 ts_manager.cc:194] Registered new tserver with Master: a8f55e88053441f48a2a3c4b65e2ca7e (127.13.160.131:40725)
I20260925 05:15:22.744150 17516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:38851
I20260925 05:15:22.751189 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:22.774639 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.767126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.13.160.129:33505)
0925 05:15:22.767303 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.767309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.767320 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:22.767349 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.767351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.767366 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.767434 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.767955 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.768711 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.769417 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.769777 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.769835 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.772149 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.772160 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.772160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.772167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.773579 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.773910 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.773913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.773913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.773968 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.774134 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.774134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.774134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.774275 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.774380 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.774442 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.774510 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:22.782579 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:22.775974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.13.160.190:40739)
0925 05:15:22.776029 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:22.776031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:22.776043 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:15:22.776154 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:22.776155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:22.776167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:22.776233 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:22.776729 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.777247 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.777950 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:22.778164 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:22.778247 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:22.780460 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:22.780461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:22.780461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:22.780468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:22.781623 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.781917 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.781918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.781918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.781966 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:22.782121 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:22.782121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:22.782122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:22.782275 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:22.782349 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:22.782430 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 05:15:22.782489 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 05:15:22.782856 17516 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33240
I20260925 05:15:22.783255 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17578
I20260925 05:15:22.788573 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17712
I20260925 05:15:22.792928 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17846
I20260925 05:15:22.797281 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 17484
2026-09-25T05:15:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1856 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17987](info): set up 2 sockets
Sep 25 05:15:22 dist-test-slave-9tgz krb5kdc[17987](info): commencing operation
krb5kdc: starting...
W20260925 05:15:23.827652 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:23 dist-test-slave-9tgz krb5kdc[17987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:23.944028 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:34703
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:34703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:24.024885 18000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:24.025142 18000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:24.025177 18000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:24.026373 18000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:24.026438 18000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:24.026461 18000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:24.026484 18000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:24.026511 18000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:24.028117 18000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:34703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:34703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:24.028398 18000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:24.028633 18000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:24.034078 18005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.034082 18008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.034294 18000 server_base.cc:1034] running on GCE node
W20260925 05:15:24.035238 18006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.035575 18000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:24.035954 18000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:24.037149 18000 hybrid_clock.cc:648] HybridClock initialized: now 1790313324037135 us; error 42 us; skew 500 ppm
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:24.044519 18000 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:24.044938 18000 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:24.044991 18000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:24.045138 18000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:24.046267 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "d18b775da2f9459c91cb166d6d9cef08"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "74aa78b5bb624fa0560d10ad0e1fa26f"
server_key_iv: "2e1c0c0b503cc235442645cfaf222a5f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.046633 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "d18b775da2f9459c91cb166d6d9cef08"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "74aa78b5bb624fa0560d10ad0e1fa26f"
server_key_iv: "2e1c0c0b503cc235442645cfaf222a5f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.048039 18000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:24.048870 18014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.049069 18000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:24.049170 18000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "d18b775da2f9459c91cb166d6d9cef08"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "74aa78b5bb624fa0560d10ad0e1fa26f"
server_key_iv: "2e1c0c0b503cc235442645cfaf222a5f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.049254 18000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:24.060541 18000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:24.062501 18000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:24.062667 18000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:24.066730 18000 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:34703
I20260925 05:15:24.066757 18066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:34703 every 8 connection(s)
I20260925 05:15:24.067094 18000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:24.068010 18067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:24.068782 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18000
I20260925 05:15:24.068908 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:24.069092 13954 external_mini_cluster.cc:1428] Setting key 5e80529f9148658a7c273a8724358845
I20260925 05:15:24.071267 18067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: Bootstrap starting.
I20260925 05:15:24.071883 18067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:24.072156 18067 log.cc:824] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:24.072892 18067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: No bootstrap required, opened a new log
I20260925 05:15:24.075464 18067 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } }
I20260925 05:15:24.075621 18067 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:24.075644 18067 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18b775da2f9459c91cb166d6d9cef08, State: Initialized, Role: FOLLOWER
I20260925 05:15:24.075779 18067 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } }
I20260925 05:15:24.075852 18067 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:24.075892 18067 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:24.075939 18067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 0 FOLLOWER]: Advancing to term 1
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:24.076593 18067 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } }
I20260925 05:15:24.076706 18067 leader_election.cc:302] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18b775da2f9459c91cb166d6d9cef08; no voters:
I20260925 05:15:24.076871 18067 leader_election.cc:288] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:24.076943 18072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:24.077153 18067 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:24.077095 18072 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [term 1 LEADER]: Becoming Leader. State: Replica: d18b775da2f9459c91cb166d6d9cef08, State: Running, Role: LEADER
I20260925 05:15:24.077453 18072 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } }
I20260925 05:15:24.078166 18073 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18b775da2f9459c91cb166d6d9cef08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } } }
I20260925 05:15:24.078258 18073 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:24.078449 18086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:24.078744 18074 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18b775da2f9459c91cb166d6d9cef08. Latest consensus state: current_term: 1 leader_uuid: "d18b775da2f9459c91cb166d6d9cef08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18b775da2f9459c91cb166d6d9cef08" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 34703 } } }
I20260925 05:15:24.078820 18074 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:24.079295 18086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:24.080647 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.069645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.13.160.190:34703)
0925 05:15:24.069804 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.069812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.069838 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:15:24.070260 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.070267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.070313 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.070428 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.071230 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.072235 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.073241 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.073681 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.073769 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.076554 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.076558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.076558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:24.076572 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.078839 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.079335 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.079338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.079339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.079458 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.079707 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.079708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.079708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.080023 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.080174 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.080290 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.080424 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260925 05:15:24.080940 18086 catalog_manager.cc:1347] Generated new cluster ID: 1cb9def3d1d24557ba74ea62d942022d
I20260925 05:15:24.080994 18086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:24.103894 18086 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:24.104457 18086 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:24.111399 18086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: Generated new TSK 0
I20260925 05:15:24.111617 18086 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:24.137171 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:24.222218 18093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:24.222451 18093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:24.222488 18093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:24.223726 18093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:24.223794 18093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:24.223842 18093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:24.225425 18093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:24.225694 18093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:24.225924 18093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:24.230991 18099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.230988 18101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.230988 18098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.231160 18093 server_base.cc:1034] running on GCE node
I20260925 05:15:24.231441 18093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:24.231699 18093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:24.232904 18093 hybrid_clock.cc:648] HybridClock initialized: now 1790313324232878 us; error 41 us; skew 500 ppm
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:24.235582 18093 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:24.235947 18093 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:24.236022 18093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:24.236188 18093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:24.237283 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "3de5e276e9cd4a6cb291000095681f4a"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "245daa754764074f3fd0a8d890cec0c2"
server_key_iv: "31bf8fad25d781ae0440e37cce0603c8"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.237644 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "3de5e276e9cd4a6cb291000095681f4a"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "245daa754764074f3fd0a8d890cec0c2"
server_key_iv: "31bf8fad25d781ae0440e37cce0603c8"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.239001 18093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:15:24.239817 18107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.240010 18093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:24.240118 18093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "3de5e276e9cd4a6cb291000095681f4a"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "245daa754764074f3fd0a8d890cec0c2"
server_key_iv: "31bf8fad25d781ae0440e37cce0603c8"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.240204 18093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:24.254453 18093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:24.256513 18093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:24.256692 18093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:24.256939 18093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:24.257257 18093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:24.257314 18093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.257354 18093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:24.257382 18093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.263361 18093 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:42373
I20260925 05:15:24.263449 18220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:42373 every 8 connection(s)
I20260925 05:15:24.263731 18093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:24.272522 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18093
I20260925 05:15:24.272610 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.264474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:36653 (local address 127.13.160.190:34703)
0925 05:15:24.264549 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.264551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.264771 (+ 220us) server_negotiation.cc:407] Connection header received
0925 05:15:24.265255 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.265256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.265265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.265320 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.266252 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.266743 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.267717 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.267913 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.267967 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.270083 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.270084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.270084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:24.270090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.271383 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.271746 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.271746 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.271747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.271795 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.272002 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.272002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.272002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.272098 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.272145 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.272399 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.272492 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 05:15:24.272631 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:24.272763 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.264468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:34703 (local address 127.13.160.129:36653)
0925 05:15:24.264627 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.264670 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.265159 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.265387 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.265391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.265656 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:24.266166 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.266171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.266825 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.266825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.267641 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.267642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.267987 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.267987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.268089 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.268654 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:24.268662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:24.269997 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:24.271456 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.271463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.271465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.271679 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.271853 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.271853 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.271854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.271950 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.272150 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:24.272154 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:24.272263 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.272359 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.272476 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260925 05:15:24.272794 13954 external_mini_cluster.cc:1428] Setting key 0e77805f6d4e2d6515fa82f2bae4eae8
I20260925 05:15:24.273217 18221 heartbeater.cc:347] Connected to a master server at 127.13.160.190:34703
I20260925 05:15:24.273312 18221 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:24.273451 18221 heartbeater.cc:511] Master 127.13.160.190:34703 requested a full tablet report, sending...
I20260925 05:15:24.274025 18031 ts_manager.cc:194] Registered new tserver with Master: 3de5e276e9cd4a6cb291000095681f4a (127.13.160.129:42373)
I20260925 05:15:24.274976 18031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:36653
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:24.298195 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:24.380692 18226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:24.380936 18226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:24.380976 18226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:24.382169 18226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:24.382236 18226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:24.382282 18226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:24.383898 18226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:24.384169 18226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:24.384399 18226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:24.389356 18231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.389499 18234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.389356 18232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.389649 18226 server_base.cc:1034] running on GCE node
I20260925 05:15:24.389860 18226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:24.390065 18226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:24.391202 18226 hybrid_clock.cc:648] HybridClock initialized: now 1790313324391190 us; error 31 us; skew 500 ppm
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:24.393680 18226 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:24.393971 18226 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:24.394013 18226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:24.394104 18226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:24.395183 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "ac93a05253004f128cd02ea9d3a5e0d0"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "599146332e77372ebf1d8607d26193b3"
server_key_iv: "708981825a43e14a8cefa8ce837b54c5"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.395483 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "ac93a05253004f128cd02ea9d3a5e0d0"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "599146332e77372ebf1d8607d26193b3"
server_key_iv: "708981825a43e14a8cefa8ce837b54c5"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.396677 18226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:24.397343 18240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.397534 18226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:24.397614 18226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "ac93a05253004f128cd02ea9d3a5e0d0"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "599146332e77372ebf1d8607d26193b3"
server_key_iv: "708981825a43e14a8cefa8ce837b54c5"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.397668 18226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:24.414912 18226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:24.417315 18226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:24.417428 18226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:24.417615 18226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:24.417949 18226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:24.417986 18226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.418008 18226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:24.418056 18226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.423765 18226 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:39041
I20260925 05:15:24.423844 18353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:39041 every 8 connection(s)
I20260925 05:15:24.424124 18226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:24.433194 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.424875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:34703 (local address 127.13.160.130:45915)
0925 05:15:24.425037 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.425072 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.425554 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.425810 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.425813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.426076 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:24.426566 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.426571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.427274 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.427275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.428071 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.428072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.428420 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.428421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.428521 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.429063 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:24.429072 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:24.430425 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:24.431926 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.431931 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.431933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.432157 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.432385 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.432385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.432386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.432482 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.432702 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:24.432706 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:24.432816 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.432896 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.433018 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260925 05:15:24.433214 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.424893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:45915 (local address 127.13.160.190:34703)
0925 05:15:24.424945 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.424947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.425173 (+ 226us) server_negotiation.cc:407] Connection header received
0925 05:15:24.425686 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.425687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.425698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.425758 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.426648 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.427158 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.428158 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.428347 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.428403 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.430566 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.430568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.430568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:24.430578 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.431857 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.432259 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.432260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.432260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.432317 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.432542 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.432543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.432543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.432648 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.432700 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.432964 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.433038 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:24.433266 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18226
I20260925 05:15:24.433360 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:24.433528 13954 external_mini_cluster.cc:1428] Setting key 73bb6c19045d1d049537ac2df84bb999
I20260925 05:15:24.434068 18354 heartbeater.cc:347] Connected to a master server at 127.13.160.190:34703
I20260925 05:15:24.434177 18354 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:24.434371 18354 heartbeater.cc:511] Master 127.13.160.190:34703 requested a full tablet report, sending...
I20260925 05:15:24.434762 18030 ts_manager.cc:194] Registered new tserver with Master: ac93a05253004f128cd02ea9d3a5e0d0 (127.13.160.130:39041)
I20260925 05:15:24.435305 18030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:45915
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:24.458200 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:34247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:24.538673 18359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:24.538875 18359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:24.538962 18359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:24.540223 18359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:24.540271 18359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:24.540299 18359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:24.541801 18359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:34247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:34703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:24.542060 18359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:24.542305 18359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:24.547153 18365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.547148 18367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.547147 18364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.547382 18359 server_base.cc:1034] running on GCE node
I20260925 05:15:24.547627 18359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:24.547861 18359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:24.549022 18359 hybrid_clock.cc:648] HybridClock initialized: now 1790313324549007 us; error 27 us; skew 500 ppm
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:24.551564 18359 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:24.551980 18359 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:24.552050 18359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:24.552187 18359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:24.553341 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "971199848dc74e0aa1237b219f5637e3"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "8d74fb40fd8131b4a1525d54a1e6ba4b"
server_key_iv: "bd0c277e7feadef909d223a860040ae4"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.553700 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "971199848dc74e0aa1237b219f5637e3"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "8d74fb40fd8131b4a1525d54a1e6ba4b"
server_key_iv: "bd0c277e7feadef909d223a860040ae4"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.555023 18359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:15:24.555763 18373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.555939 18359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:24.556036 18359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "971199848dc74e0aa1237b219f5637e3"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "8d74fb40fd8131b4a1525d54a1e6ba4b"
server_key_iv: "bd0c277e7feadef909d223a860040ae4"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.556123 18359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:24.579242 18359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:24.581406 18359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:24.581583 18359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:24.581836 18359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:24.582158 18359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:24.582214 18359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.582250 18359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:24.582279 18359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.588102 18359 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:37395
I20260925 05:15:24.588137 18486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:37395 every 8 connection(s)
I20260925 05:15:24.588428 18359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:24.593103 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18359
I20260925 05:15:24.593175 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:24.593353 13954 external_mini_cluster.cc:1428] Setting key a75ed16ad7ab1b9e8b78777e8bcc9061
Sep 25 05:15:24 dist-test-slave-9tgz krb5kdc[17987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313324, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:24.597332 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.589194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:45837 (local address 127.13.160.190:34703)
0925 05:15:24.589242 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.589245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.589442 (+ 197us) server_negotiation.cc:407] Connection header received
0925 05:15:24.589902 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.589903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.589914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.589966 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.590870 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.591413 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.592378 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.592573 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.592623 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.595045 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.595046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.595046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:24.595053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.596216 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.596567 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.596568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.596568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.596620 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.596821 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.596822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.596822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.596922 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.596969 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.597206 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.597266 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:24.597513 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.589139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:34703 (local address 127.13.160.131:45837)
0925 05:15:24.589318 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.589355 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.589821 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.590021 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.590025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.590298 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:24.590796 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.590800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.591494 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.591495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.592306 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.592308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.592641 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.592641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.592738 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.593297 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:24.593305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:24.594961 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:24.596283 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.596288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.596290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.596503 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.596677 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:24.596678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:24.596678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:24.596772 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:24.596974 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:24.596978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:24.597102 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.597182 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.597299 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":69,"threads_started":1}
I20260925 05:15:24.597977 18487 heartbeater.cc:347] Connected to a master server at 127.13.160.190:34703
I20260925 05:15:24.598095 18487 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:24.598259 18487 heartbeater.cc:511] Master 127.13.160.190:34703 requested a full tablet report, sending...
I20260925 05:15:24.598632 18030 ts_manager.cc:194] Registered new tserver with Master: 971199848dc74e0aa1237b219f5637e3 (127.13.160.131:37395)
I20260925 05:15:24.599078 18030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:45837
I20260925 05:15:24.605046 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:24.612341 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.606298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.13.160.190:34703)
0925 05:15:24.606369 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.606371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.606439 (+ 68us) server_negotiation.cc:407] Connection header received
0925 05:15:24.606479 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.606480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.606488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.606535 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.607143 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.607623 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.608280 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.608453 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.608503 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.609596 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.609597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.609597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:24.609603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.611062 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.611485 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.611487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.611487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.611568 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:24.611771 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:24.611772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:24.611773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:24.611931 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.612034 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.612163 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.612239 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 05:15:24.613744 18030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:24.615350 18030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:24.621711 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:42373 (local address 127.0.0.1:53302)
0925 05:15:24.617920 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.617931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.617977 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.618187 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.618191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.618193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:24.618271 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.618274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619414 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.619415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.620601 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.620603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621411 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.621491 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.621504 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.621575 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.621630 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":91,"threads_started":1}
I20260925 05:15:24.621723 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.13.160.129:42373)
0925 05:15:24.617997 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.618002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.618013 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:24.618052 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.618054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.618069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.618175 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:24.618366 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619312 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.620687 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621323 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621413 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.621472 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.621607 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.621642 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260925 05:15:24.621990 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37395 (local address 127.0.0.1:38920)
0925 05:15:24.617683 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.617723 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.617793 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.618398 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.618399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.618400 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:24.618454 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.618455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619812 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.619813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.620837 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.620838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621757 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.621821 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.621826 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.621858 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.621896 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":62,"threads_started":1}
I20260925 05:15:24.622186 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.13.160.130:39041)
0925 05:15:24.618341 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.618345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.618354 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:24.618389 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.618391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.618407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.618558 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:24.618783 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619834 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621041 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621707 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621768 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.621821 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.622044 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.622091 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":85,"threads_started":1}
I20260925 05:15:24.622187 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39041 (local address 127.0.0.1:54552)
0925 05:15:24.618236 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.618246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.618310 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.618650 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.618651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.618653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:24.618704 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.618705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619937 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.619938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.620956 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.620959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621851 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.621852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.621913 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.621916 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.621961 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.621991 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260925 05:15:24.622395 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.617748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38920 (local address 127.13.160.131:37395)
0925 05:15:24.618034 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.618038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.618047 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:24.618083 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.618091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.618103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.618526 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:24.618617 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.619706 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.620916 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.621650 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.622187 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.622247 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.622276 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.622320 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":92,"threads_started":1}
I20260925 05:15:24.622802 18155 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae63d7e8fee404faf908593c8737293 (DEFAULT_TABLE table=test-table [id=8d5ee7193c3a415389177d99297d3948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:24.623070 18421 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae63d7e8fee404faf908593c8737293 (DEFAULT_TABLE table=test-table [id=8d5ee7193c3a415389177d99297d3948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:24.623237 18155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae63d7e8fee404faf908593c8737293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:24.623406 18421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae63d7e8fee404faf908593c8737293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:24.625917 18504 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Bootstrap starting.
I20260925 05:15:24.626332 18505 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Bootstrap starting.
I20260925 05:15:24.626612 18504 tablet_bootstrap.cc:654] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:24.626509 18288 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae63d7e8fee404faf908593c8737293 (DEFAULT_TABLE table=test-table [id=8d5ee7193c3a415389177d99297d3948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:24.626856 18288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae63d7e8fee404faf908593c8737293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:24.626973 18505 tablet_bootstrap.cc:654] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:24.627264 18504 log.cc:824] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:24.627516 18505 log.cc:824] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:24.628031 18504 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: No bootstrap required, opened a new log
I20260925 05:15:24.628100 18504 ts_tablet_manager.cc:1397] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:15:24.628242 18505 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: No bootstrap required, opened a new log
I20260925 05:15:24.628288 18505 ts_tablet_manager.cc:1397] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:24.629781 18508 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Bootstrap starting.
I20260925 05:15:24.630105 18504 raft_consensus.cc:348] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.630254 18508 tablet_bootstrap.cc:654] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:24.630167 18505 raft_consensus.cc:348] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.630263 18504 raft_consensus.cc:374] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:24.630287 18505 raft_consensus.cc:374] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:24.630299 18504 raft_consensus.cc:729] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de5e276e9cd4a6cb291000095681f4a, State: Initialized, Role: FOLLOWER
I20260925 05:15:24.630321 18505 raft_consensus.cc:729] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 971199848dc74e0aa1237b219f5637e3, State: Initialized, Role: FOLLOWER
I20260925 05:15:24.630420 18504 consensus_queue.cc:260] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.630420 18505 consensus_queue.cc:260] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.630697 18505 ts_tablet_manager.cc:1428] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 05:15:24.630715 18504 ts_tablet_manager.cc:1428] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 05:15:24.630748 18221 heartbeater.cc:503] Master 127.13.160.190:34703 was elected leader, sending a full tablet report...
I20260925 05:15:24.630807 18487 heartbeater.cc:503] Master 127.13.160.190:34703 was elected leader, sending a full tablet report...
I20260925 05:15:24.630827 18508 log.cc:824] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:24.631572 18508 tablet_bootstrap.cc:492] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: No bootstrap required, opened a new log
I20260925 05:15:24.631644 18508 ts_tablet_manager.cc:1397] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:24.633136 18508 raft_consensus.cc:348] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.633320 18508 raft_consensus.cc:374] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:24.633358 18508 raft_consensus.cc:729] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac93a05253004f128cd02ea9d3a5e0d0, State: Initialized, Role: FOLLOWER
I20260925 05:15:24.633484 18508 consensus_queue.cc:260] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.633762 18508 ts_tablet_manager.cc:1428] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:24.633788 18354 heartbeater.cc:503] Master 127.13.160.190:34703 was elected leader, sending a full tablet report...
W20260925 05:15:24.674765 18355 tablet.cc:2369] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:15:24.764405 18222 tablet.cc:2369] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:24.774307 18512 raft_consensus.cc:482] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:24.774474 18512 raft_consensus.cc:504] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.774873 18512 leader_election.cc:288] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 971199848dc74e0aa1237b219f5637e3 (127.13.160.131:37395), 3de5e276e9cd4a6cb291000095681f4a (127.13.160.129:42373)
I20260925 05:15:24.777909 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.775145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:42373 (local address 127.13.160.130:46351)
0925 05:15:24.775312 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.775321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.775370 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.775524 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.775525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.775527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:24.775599 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.775601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.776215 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.776216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.776974 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.776976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.777650 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.777728 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.777733 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.777788 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.777825 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":4,"thread_start_us":115,"threads_started":1}
I20260925 05:15:24.777937 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.775203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:46351 (local address 127.13.160.129:42373)
0925 05:15:24.775252 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.775255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.775385 (+ 130us) server_negotiation.cc:407] Connection header received
0925 05:15:24.775432 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.775433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.775442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.775508 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:24.775665 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.776137 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777083 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.777568 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777632 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.777639 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.777811 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.777850 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:15:24.778017 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.775072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:37395 (local address 127.13.160.130:33273)
0925 05:15:24.775124 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.775138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.775188 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.775347 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:24.775348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:24.775350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:24.775446 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.775448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.776147 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.776149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.776989 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:24.776991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.777750 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:24.777854 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.777857 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:24.777915 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 05:15:24.777949 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":48}
I20260925 05:15:24.778038 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.775168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:33273 (local address 127.13.160.131:37395)
0925 05:15:24.775215 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.775217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.775238 (+ 21us) server_negotiation.cc:407] Connection header received
0925 05:15:24.775265 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.775266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.775276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.775330 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:24.775540 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.776065 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777132 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.777673 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.777745 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.777754 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.777934 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.777976 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:15:24.778350 18441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae63d7e8fee404faf908593c8737293" candidate_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "971199848dc74e0aa1237b219f5637e3" is_pre_election: true
I20260925 05:15:24.778373 18175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae63d7e8fee404faf908593c8737293" candidate_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de5e276e9cd4a6cb291000095681f4a" is_pre_election: true
I20260925 05:15:24.778509 18175 raft_consensus.cc:2454] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac93a05253004f128cd02ea9d3a5e0d0 in term 0.
I20260925 05:15:24.778509 18441 raft_consensus.cc:2454] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac93a05253004f128cd02ea9d3a5e0d0 in term 0.
I20260925 05:15:24.778725 18243 leader_election.cc:302] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de5e276e9cd4a6cb291000095681f4a, ac93a05253004f128cd02ea9d3a5e0d0; no voters:
I20260925 05:15:24.778869 18512 raft_consensus.cc:2781] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:24.778941 18512 raft_consensus.cc:482] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:24.778970 18512 raft_consensus.cc:3037] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:24.779627 18512 raft_consensus.cc:504] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.779775 18512 leader_election.cc:288] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [CANDIDATE]: Term 1 election: Requested vote from peers 971199848dc74e0aa1237b219f5637e3 (127.13.160.131:37395), 3de5e276e9cd4a6cb291000095681f4a (127.13.160.129:42373)
I20260925 05:15:24.779928 18175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae63d7e8fee404faf908593c8737293" candidate_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de5e276e9cd4a6cb291000095681f4a"
I20260925 05:15:24.779938 18441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae63d7e8fee404faf908593c8737293" candidate_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "971199848dc74e0aa1237b219f5637e3"
I20260925 05:15:24.780011 18441 raft_consensus.cc:3037] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:24.780033 18175 raft_consensus.cc:3037] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:24.780735 18441 raft_consensus.cc:2454] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac93a05253004f128cd02ea9d3a5e0d0 in term 1.
I20260925 05:15:24.780753 18175 raft_consensus.cc:2454] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac93a05253004f128cd02ea9d3a5e0d0 in term 1.
I20260925 05:15:24.780910 18244 leader_election.cc:302] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 971199848dc74e0aa1237b219f5637e3, ac93a05253004f128cd02ea9d3a5e0d0; no voters:
I20260925 05:15:24.781080 18512 raft_consensus.cc:2781] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:24.781270 18512 raft_consensus.cc:686] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 1 LEADER]: Becoming Leader. State: Replica: ac93a05253004f128cd02ea9d3a5e0d0, State: Running, Role: LEADER
I20260925 05:15:24.781386 18512 consensus_queue.cc:237] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } }
I20260925 05:15:24.782258 18031 catalog_manager.cc:5495] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac93a05253004f128cd02ea9d3a5e0d0 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac93a05253004f128cd02ea9d3a5e0d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39041 } health_report { overall_health: HEALTHY } } }
I20260925 05:15:24.802968 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.800743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.13.160.130:39041)
0925 05:15:24.800789 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.800793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.800804 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:24.800837 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.800838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.800855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.800911 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:24.801060 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.801521 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.802244 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:24.802420 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:24.802469 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:24.802607 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:24.802706 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:24.802766 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.802794 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.802811 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:24.804767 18175 raft_consensus.cc:1261] T 8ae63d7e8fee404faf908593c8737293 P 3de5e276e9cd4a6cb291000095681f4a [term 1 FOLLOWER]: Refusing update from remote peer ac93a05253004f128cd02ea9d3a5e0d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:24.804775 18441 raft_consensus.cc:1261] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 1 FOLLOWER]: Refusing update from remote peer ac93a05253004f128cd02ea9d3a5e0d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:24.805162 18514 consensus_queue.cc:1035] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de5e276e9cd4a6cb291000095681f4a" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 42373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:24.805339 18512 consensus_queue.cc:1035] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "971199848dc74e0aa1237b219f5637e3" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:24.808187 18517 mvcc.cc:205] Tried to move back new op lower bound from 7333123378397462528 to 7333123378305126400. Current Snapshot: MvccSnapshot[applied={T|T < 7333123378397462528}]
I20260925 05:15:24.808463 18518 mvcc.cc:205] Tried to move back new op lower bound from 7333123378397462528 to 7333123378305126400. Current Snapshot: MvccSnapshot[applied={T|T < 7333123378397462528 or (T in {7333123378397462528})}]
I20260925 05:15:24.810298 18030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:24.810384 18030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:24.811151 18030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d18b775da2f9459c91cb166d6d9cef08: Sending DeleteTablet for 3 replicas of tablet 8ae63d7e8fee404faf908593c8737293
I20260925 05:15:24.811460 18421 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae63d7e8fee404faf908593c8737293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:24 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:38920
I20260925 05:15:24.811462 18155 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae63d7e8fee404faf908593c8737293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:24 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:53302
I20260925 05:15:24.811803 18288 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae63d7e8fee404faf908593c8737293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:24 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:54552
I20260925 05:15:24.811941 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18093
I20260925 05:15:24.812507 18528 tablet_replica.cc:330] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: stopping tablet replica
I20260925 05:15:24.812624 18528 raft_consensus.cc:2229] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:24.812727 18528 raft_consensus.cc:2258] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:24.813218 18528 ts_tablet_manager.cc:1905] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:24.815207 18528 ts_tablet_manager.cc:1918] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:24.815275 18528 log.cc:1196] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/8ae63d7e8fee404faf908593c8737293
I20260925 05:15:24.815562 18528 ts_tablet_manager.cc:1939] T 8ae63d7e8fee404faf908593c8737293 P 971199848dc74e0aa1237b219f5637e3: Deleting consensus metadata
I20260925 05:15:24.815974 18018 catalog_manager.cc:4841] TS 971199848dc74e0aa1237b219f5637e3 (127.13.160.131:37395): tablet 8ae63d7e8fee404faf908593c8737293 (table test-table [id=8d5ee7193c3a415389177d99297d3948]) successfully deleted
W20260925 05:15:24.818511 18017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:42373 (error 108)
I20260925 05:15:24.819015 18529 tablet_replica.cc:330] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0: stopping tablet replica
W20260925 05:15:24.819097 18500 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:24.818866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:42373 (local address 127.0.0.1:53306)
0925 05:15:24.818934 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.818948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:24.818997 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:24.819049 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:42373: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 05:15:24.819180 18529 raft_consensus.cc:2229] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 1 LEADER]: Raft consensus shutting down.
W20260925 05:15:24.819222 18017 catalog_manager.cc:4584] TS 3de5e276e9cd4a6cb291000095681f4a (127.13.160.129:42373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ae63d7e8fee404faf908593c8737293: Network error: Client connection negotiation failed: client connection to 127.13.160.129:42373: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260925 05:15:24.819309 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18226
I20260925 05:15:24.819344 18529 raft_consensus.cc:2258] T 8ae63d7e8fee404faf908593c8737293 P ac93a05253004f128cd02ea9d3a5e0d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:24.823644 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18359
I20260925 05:15:24.823781 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.823627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39041 (local address 127.0.0.1:54576)
0925 05:15:24.823684 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:24.823725 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 05:15:24.827586 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18000
2026-09-25T05:15:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2030 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T05:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:24Z Disabled control of system clock
I20260925 05:15:24.854763 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36425
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:36425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 05:15:24.936141 18534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:24.936355 18534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:24.936376 18534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:24.937456 18534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:24.937491 18534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:24.937505 18534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:24.937516 18534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:24.937528 18534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:24.938884 18534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:36425
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:24.939152 18534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:24.939394 18534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:24.944146 18542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.944198 18540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:24.944162 18539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:24.944701 18534 server_base.cc:1034] running on GCE node
I20260925 05:15:24.944909 18534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:24.945181 18534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:24.946369 18534 hybrid_clock.cc:648] HybridClock initialized: now 1790313324946356 us; error 25 us; skew 500 ppm
I20260925 05:15:24.947643 18534 webserver.cc:460] Webserver started at http://127.13.160.190:41235/ using document root <none> and password file <none>
I20260925 05:15:24.947902 18534 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:24.947971 18534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:24.948133 18534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:24.949218 18534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "8a14a545633a42749fd8a672fa26289b"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "5fbc7d9e1100922a23428999181fcfca"
server_key_iv: "1815a93a29bae8e9afecbb404edc835f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.949594 18534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "8a14a545633a42749fd8a672fa26289b"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "5fbc7d9e1100922a23428999181fcfca"
server_key_iv: "1815a93a29bae8e9afecbb404edc835f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.950956 18534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:24.951709 18548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:24.951879 18534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:24.951972 18534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "8a14a545633a42749fd8a672fa26289b"
format_stamp: "Formatted at 2026-09-25 05:15:24 on dist-test-slave-9tgz"
server_key: "5fbc7d9e1100922a23428999181fcfca"
server_key_iv: "1815a93a29bae8e9afecbb404edc835f"
server_key_version: "encryptionkey@0"
I20260925 05:15:24.952056 18534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:24.955578 18534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:24.955957 18534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:24.956105 18534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:24.960208 18534 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:36425
I20260925 05:15:24.960285 18600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:36425 every 8 connection(s)
I20260925 05:15:24.960652 18534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:24.961338 18601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:24.964303 18601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: Bootstrap starting.
I20260925 05:15:24.964859 18601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:24.965097 18601 log.cc:824] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:24.965768 18601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: No bootstrap required, opened a new log
I20260925 05:15:24.967379 18601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } }
I20260925 05:15:24.967526 18601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:24.967549 18601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a14a545633a42749fd8a672fa26289b, State: Initialized, Role: FOLLOWER
I20260925 05:15:24.967682 18601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } }
I20260925 05:15:24.967758 18601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:24.967780 18601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:24.967808 18601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:24.968314 18601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } }
I20260925 05:15:24.968398 18601 leader_election.cc:302] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a14a545633a42749fd8a672fa26289b; no voters:
I20260925 05:15:24.968533 18601 leader_election.cc:288] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:24.968621 18604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:24.968796 18604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [term 1 LEADER]: Becoming Leader. State: Replica: 8a14a545633a42749fd8a672fa26289b, State: Running, Role: LEADER
I20260925 05:15:24.968816 18601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:24.968887 18604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } }
I20260925 05:15:24.968945 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18534
I20260925 05:15:24.969053 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:24.969271 13954 external_mini_cluster.cc:1428] Setting key 759657b43b2ab8000968a3b33235e5e0
I20260925 05:15:24.969233 18604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a14a545633a42749fd8a672fa26289b. Latest consensus state: current_term: 1 leader_uuid: "8a14a545633a42749fd8a672fa26289b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } } }
I20260925 05:15:24.969312 18604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:24.969477 18605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a14a545633a42749fd8a672fa26289b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a14a545633a42749fd8a672fa26289b" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36425 } } }
I20260925 05:15:24.969522 18611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:24.969543 18605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:24.970005 18611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:24.971889 18611 catalog_manager.cc:1347] Generated new cluster ID: 6106acb6242e41879b0a936f0ba00cd2
I20260925 05:15:24.971944 18611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:24.972177 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:24.969926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.13.160.190:36425)
0925 05:15:24.970489 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:24.970497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:24.970521 (+ 24us) server_negotiation.cc:407] Connection header received
0925 05:15:24.970724 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:24.970734 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:24.970794 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:24.971017 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:24.971700 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:24.971704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:24.971704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:24.971709 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:24.971756 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:24.971767 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:24.971836 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:24.971944 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 05:15:24.972018 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":78,"threads_started":1}
I20260925 05:15:24.984771 18611 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:24.984957 18611 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:24.994630 18611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: Generated new TSK 0
I20260925 05:15:24.994864 18611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:25.001421 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--builtin_ntp_servers=127.13.160.148:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 05:15:25.084888 18625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:25.085100 18625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:25.085124 18625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:25.086304 18625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:25.086344 18625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:25.086369 18625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:25.087791 18625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:25.088055 18625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:25.088361 18625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:25.093503 18630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.093529 18631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.093562 18625 server_base.cc:1034] running on GCE node
W20260925 05:15:25.093713 18633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.093991 18625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:25.094264 18625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:25.095423 18625 hybrid_clock.cc:648] HybridClock initialized: now 1790313325095412 us; error 32 us; skew 500 ppm
I20260925 05:15:25.096684 18625 webserver.cc:460] Webserver started at http://127.13.160.129:41083/ using document root <none> and password file <none>
I20260925 05:15:25.096941 18625 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:25.097013 18625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:25.097165 18625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:25.098240 18625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "d0df1ee0f6344fa398d26c764eca485d"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "58cf469f45f69155ad396f2d7133629a"
server_key_iv: "3952680341ce0650b4a9e255e5d204c4"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.098596 18625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "d0df1ee0f6344fa398d26c764eca485d"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "58cf469f45f69155ad396f2d7133629a"
server_key_iv: "3952680341ce0650b4a9e255e5d204c4"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.099996 18625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:25.100739 18639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.100908 18625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:25.101011 18625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "d0df1ee0f6344fa398d26c764eca485d"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "58cf469f45f69155ad396f2d7133629a"
server_key_iv: "3952680341ce0650b4a9e255e5d204c4"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.101090 18625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:25.104444 18625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:25.104710 18625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:25.104848 18625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:25.105055 18625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:25.105360 18625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:25.105412 18625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.105450 18625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:25.105477 18625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.111004 18625 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:36315
I20260925 05:15:25.111045 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:36315 every 8 connection(s)
I20260925 05:15:25.111367 18625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:25.114274 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.112091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:56463 (local address 127.13.160.190:36425)
0925 05:15:25.112138 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.112140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.112402 (+ 262us) server_negotiation.cc:407] Connection header received
0925 05:15:25.112849 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.112850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.112855 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.112918 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.114052 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.114053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.114054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.114057 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.114072 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.114076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.114122 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.114208 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.114221 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:25.114471 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.112064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36425 (local address 127.13.160.129:56463)
0925 05:15:25.112265 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.112305 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.112762 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.112999 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.113003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.113011 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.113903 (+ 892us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.113903 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.113931 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.113959 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.114134 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.114138 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.114140 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.114200 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.114248 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260925 05:15:25.115018 18753 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36425
I20260925 05:15:25.115104 18753 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:25.115266 18753 heartbeater.cc:511] Master 127.13.160.190:36425 requested a full tablet report, sending...
I20260925 05:15:25.115783 18565 ts_manager.cc:194] Registered new tserver with Master: d0df1ee0f6344fa398d26c764eca485d (127.13.160.129:36315)
I20260925 05:15:25.116436 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18625
I20260925 05:15:25.116524 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:25.116721 13954 external_mini_cluster.cc:1428] Setting key 72e56cb56fdcbb7f871345075b1948b0
I20260925 05:15:25.117566 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--builtin_ntp_servers=127.13.160.148:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 05:15:25.199282 18756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:25.199506 18756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:25.199530 18756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:25.200670 18756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:25.200706 18756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:25.200731 18756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:25.201994 18756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:25.202198 18756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:25.202448 18756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:25.207406 18761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.207346 18764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.207350 18762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.207631 18756 server_base.cc:1034] running on GCE node
I20260925 05:15:25.207851 18756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:25.208078 18756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:25.209234 18756 hybrid_clock.cc:648] HybridClock initialized: now 1790313325209212 us; error 32 us; skew 500 ppm
I20260925 05:15:25.210419 18756 webserver.cc:460] Webserver started at http://127.13.160.130:34493/ using document root <none> and password file <none>
I20260925 05:15:25.210673 18756 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:25.210721 18756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:25.210868 18756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:25.212049 18756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "20d65887628e4bd1ac35dd2c794bb494"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "81c8444a5a688c87582084c8c270f602"
server_key_iv: "dacee0df0cdc851519bec9ee6b9a2b3e"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.212409 18756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "20d65887628e4bd1ac35dd2c794bb494"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "81c8444a5a688c87582084c8c270f602"
server_key_iv: "dacee0df0cdc851519bec9ee6b9a2b3e"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.213701 18756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:25.214378 18770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.214552 18756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:25.214648 18756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "20d65887628e4bd1ac35dd2c794bb494"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "81c8444a5a688c87582084c8c270f602"
server_key_iv: "dacee0df0cdc851519bec9ee6b9a2b3e"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.214727 18756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:25.218062 18756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:25.218528 18756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:25.218709 18756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:25.218958 18756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:25.219262 18756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:25.219317 18756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.219357 18756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:25.219384 18756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.225003 18756 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:35595
I20260925 05:15:25.225072 18883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:35595 every 8 connection(s)
I20260925 05:15:25.225412 18756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:25.228543 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.226161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:40067 (local address 127.13.160.190:36425)
0925 05:15:25.226205 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.226208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.226421 (+ 213us) server_negotiation.cc:407] Connection header received
0925 05:15:25.226989 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.226990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.226998 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.227060 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.228269 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.228271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.228272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.228277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.228297 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.228301 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.228362 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.228456 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.228472 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:15:25.228760 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.226129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36425 (local address 127.13.160.130:40067)
0925 05:15:25.226296 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.226328 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.226831 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.227133 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.227137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.227147 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.228115 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.228115 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.228143 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.228170 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.228364 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.228369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.228373 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.228466 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.228534 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260925 05:15:25.229218 18884 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36425
I20260925 05:15:25.229312 18884 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:25.229462 18884 heartbeater.cc:511] Master 127.13.160.190:36425 requested a full tablet report, sending...
I20260925 05:15:25.229802 18565 ts_manager.cc:194] Registered new tserver with Master: 20d65887628e4bd1ac35dd2c794bb494 (127.13.160.130:35595)
I20260925 05:15:25.231937 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18756
I20260925 05:15:25.232048 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:25.232229 13954 external_mini_cluster.cc:1428] Setting key abe26e607042a6ad720aaee2e85adc28
I20260925 05:15:25.233062 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--builtin_ntp_servers=127.13.160.148:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 05:15:25.313771 18887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:25.313982 18887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:25.314003 18887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:25.315231 18887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:25.315274 18887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:25.315299 18887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:25.316669 18887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:25.316920 18887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:25.317276 18887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:25.322160 18895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.322196 18892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.322184 18893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.322486 18887 server_base.cc:1034] running on GCE node
I20260925 05:15:25.322731 18887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:25.323009 18887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:25.324170 18887 hybrid_clock.cc:648] HybridClock initialized: now 1790313325324155 us; error 44 us; skew 500 ppm
I20260925 05:15:25.325419 18887 webserver.cc:460] Webserver started at http://127.13.160.131:46587/ using document root <none> and password file <none>
I20260925 05:15:25.325685 18887 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:25.325757 18887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:25.325915 18887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:25.327090 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "15c41a63a3a141f4be4136b6cfce9cf4"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "3fd7902a0db5e244c4d06fa561db1df0"
server_key_iv: "88e18b375f0e08adcd14d9e302cebb74"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.327469 18887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "15c41a63a3a141f4be4136b6cfce9cf4"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "3fd7902a0db5e244c4d06fa561db1df0"
server_key_iv: "88e18b375f0e08adcd14d9e302cebb74"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.328747 18887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:25.329504 18901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.329687 18887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:25.329784 18887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "15c41a63a3a141f4be4136b6cfce9cf4"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "3fd7902a0db5e244c4d06fa561db1df0"
server_key_iv: "88e18b375f0e08adcd14d9e302cebb74"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.329867 18887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:25.333271 18887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:25.333540 18887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:25.333689 18887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:25.333885 18887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:25.334206 18887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:25.334260 18887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.334298 18887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:25.334326 18887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.339931 18887 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:40263
I20260925 05:15:25.340005 19014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:40263 every 8 connection(s)
I20260925 05:15:25.340330 18887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:25.343286 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.341040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:52669 (local address 127.13.160.190:36425)
0925 05:15:25.341093 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.341096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.341392 (+ 296us) server_negotiation.cc:407] Connection header received
0925 05:15:25.341837 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.341838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.341844 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.341898 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.343065 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.343067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.343067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.343071 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.343085 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.343089 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.343141 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.343220 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.343233 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:15:25.343510 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.341064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36425 (local address 127.13.160.131:52669)
0925 05:15:25.341275 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.341308 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.341757 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.341951 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.341955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.341963 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.342848 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.342849 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.342878 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.342971 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.343149 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.343154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.343156 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.343216 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.343267 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260925 05:15:25.344028 19015 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36425
I20260925 05:15:25.344144 19015 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:25.344293 19015 heartbeater.cc:511] Master 127.13.160.190:36425 requested a full tablet report, sending...
I20260925 05:15:25.344664 18565 ts_manager.cc:194] Registered new tserver with Master: 15c41a63a3a141f4be4136b6cfce9cf4 (127.13.160.131:40263)
I20260925 05:15:25.347625 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 18887
I20260925 05:15:25.347711 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:25.347842 13954 external_mini_cluster.cc:1428] Setting key 15fdba00279fc86eeefa458f4bf137da
I20260925 05:15:25.348702 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:25.351153 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.349947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.13.160.190:36425)
0925 05:15:25.350026 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.350029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.350108 (+ 79us) server_negotiation.cc:407] Connection header received
0925 05:15:25.350190 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.350190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.350196 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.350252 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.350921 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.350923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.350923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.350928 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.350945 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.350950 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.351007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.351066 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.351081 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 05:15:25.352778 18565 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:25.354272 18565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:25.358119 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:36315 (local address 127.0.0.1:57744)
0925 05:15:25.356512 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.356551 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.356603 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.357136 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.357141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.357166 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.357661 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.357661 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.357679 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.357706 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.357943 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.357950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.357953 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.358025 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.358040 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260925 05:15:25.358206 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.13.160.129:36315)
0925 05:15:25.356664 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.356669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.356680 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:25.356710 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.356712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.356723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.356808 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.357803 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.357813 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.357814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.357819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.357855 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.357891 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.357950 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.358089 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.358107 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260925 05:15:25.358244 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:35595 (local address 127.0.0.1:43310)
0925 05:15:25.356909 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.356919 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.356960 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.357356 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.357357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.357360 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.357837 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.357837 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.357844 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.357856 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.358110 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.358111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.358112 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.358163 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.358175 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260925 05:15:25.358664 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40263 (local address 127.0.0.1:49688)
0925 05:15:25.356733 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.356744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.356785 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.357568 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.357569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.357571 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.358312 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.358312 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.358318 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.358328 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.358553 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.358554 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.358555 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.358595 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.358607 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260925 05:15:25.358784 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.13.160.130:35595)
0925 05:15:25.357212 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.357216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.357227 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:25.357250 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.357252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.357260 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.357518 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.357940 (+ 422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.357945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.357946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.357950 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.357982 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.358001 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.358059 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.358464 (+ 405us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.358484 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":84,"threads_started":1}
I20260925 05:15:25.358883 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.356990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.13.160.131:40263)
0925 05:15:25.357150 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.357155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.357166 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:25.357191 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.357193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.357202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.357279 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.358401 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.358409 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.358409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.358413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.358447 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.358473 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.358527 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.358767 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.358785 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260925 05:15:25.358932 18687 tablet_service.cc:1467] Processing CreateTablet for tablet 494a147d774042e58f8ecb88db3da755 (DEFAULT_TABLE table=test-table [id=2b1cdc81244a4353a54410d3398ec70b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:25.359333 18687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a147d774042e58f8ecb88db3da755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:25.359512 18949 tablet_service.cc:1467] Processing CreateTablet for tablet 494a147d774042e58f8ecb88db3da755 (DEFAULT_TABLE table=test-table [id=2b1cdc81244a4353a54410d3398ec70b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:25.359828 18949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a147d774042e58f8ecb88db3da755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:25.359722 18818 tablet_service.cc:1467] Processing CreateTablet for tablet 494a147d774042e58f8ecb88db3da755 (DEFAULT_TABLE table=test-table [id=2b1cdc81244a4353a54410d3398ec70b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:25.360064 18818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a147d774042e58f8ecb88db3da755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:25.362517 19029 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Bootstrap starting.
I20260925 05:15:25.363065 19030 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: Bootstrap starting.
I20260925 05:15:25.363160 19031 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Bootstrap starting.
I20260925 05:15:25.363502 19029 tablet_bootstrap.cc:654] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:25.363729 19031 tablet_bootstrap.cc:654] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:25.363798 19030 tablet_bootstrap.cc:654] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:25.363925 19029 log.cc:824] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:25.364043 19031 log.cc:824] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:25.364113 19030 log.cc:824] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:25.364761 19031 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: No bootstrap required, opened a new log
I20260925 05:15:25.364795 19029 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: No bootstrap required, opened a new log
I20260925 05:15:25.364833 19031 ts_tablet_manager.cc:1397] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:25.364845 19029 ts_tablet_manager.cc:1397] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:25.367237 19030 tablet_bootstrap.cc:492] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: No bootstrap required, opened a new log
I20260925 05:15:25.367323 19030 ts_tablet_manager.cc:1397] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 05:15:25.367516 19029 raft_consensus.cc:348] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.367516 19031 raft_consensus.cc:348] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.367692 19031 raft_consensus.cc:374] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:25.367691 19029 raft_consensus.cc:374] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:25.367727 19031 raft_consensus.cc:729] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c41a63a3a141f4be4136b6cfce9cf4, State: Initialized, Role: FOLLOWER
I20260925 05:15:25.367727 19029 raft_consensus.cc:729] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20d65887628e4bd1ac35dd2c794bb494, State: Initialized, Role: FOLLOWER
I20260925 05:15:25.367851 19031 consensus_queue.cc:260] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.367897 19029 consensus_queue.cc:260] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.368114 19031 ts_tablet_manager.cc:1428] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 05:15:25.368120 19029 ts_tablet_manager.cc:1428] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 05:15:25.368237 19015 heartbeater.cc:503] Master 127.13.160.190:36425 was elected leader, sending a full tablet report...
I20260925 05:15:25.368575 18884 heartbeater.cc:503] Master 127.13.160.190:36425 was elected leader, sending a full tablet report...
I20260925 05:15:25.369983 19030 raft_consensus.cc:348] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.370108 19030 raft_consensus.cc:374] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:25.370162 19030 raft_consensus.cc:729] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0df1ee0f6344fa398d26c764eca485d, State: Initialized, Role: FOLLOWER
I20260925 05:15:25.370281 19030 consensus_queue.cc:260] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.370528 19030 ts_tablet_manager.cc:1428] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:25.370559 18753 heartbeater.cc:503] Master 127.13.160.190:36425 was elected leader, sending a full tablet report...
W20260925 05:15:25.475960 18885 tablet.cc:2369] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:25.520699 19036 raft_consensus.cc:482] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:25.520874 19036 raft_consensus.cc:504] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.521265 19036 leader_election.cc:288] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0df1ee0f6344fa398d26c764eca485d (127.13.160.129:36315), 15c41a63a3a141f4be4136b6cfce9cf4 (127.13.160.131:40263)
I20260925 05:15:25.522693 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.521629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:54111 (local address 127.13.160.129:36315)
0925 05:15:25.521680 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.521682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.521695 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:15:25.521726 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.521727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.521733 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.521788 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.522445 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.522447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.522447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.522452 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.522473 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.522477 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.522542 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.522611 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.522626 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 05:15:25.522693 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.521514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:36315 (local address 127.13.160.130:54111)
0925 05:15:25.521589 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.521604 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.521671 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.521823 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.521824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.521827 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.522341 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.522341 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.522348 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.522359 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.522538 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.522538 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.522539 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.522610 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.522626 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 05:15:25.522996 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.521762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:48433 (local address 127.13.160.131:40263)
0925 05:15:25.521809 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.521811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.521857 (+ 46us) server_negotiation.cc:407] Connection header received
0925 05:15:25.521909 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.521910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.521915 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.521983 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.522651 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.522652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.522653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.522658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.522674 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.522678 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.522777 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.522877 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.522892 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:25.523065 18707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a147d774042e58f8ecb88db3da755" candidate_uuid: "20d65887628e4bd1ac35dd2c794bb494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0df1ee0f6344fa398d26c764eca485d" is_pre_election: true
I20260925 05:15:25.523205 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.521663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40263 (local address 127.13.160.130:48433)
0925 05:15:25.521807 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.521816 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.521866 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.522014 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.522014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.522017 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:25.522543 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:25.522543 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:25.522549 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:25.522559 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:25.522798 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:25.522799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:25.522800 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 05:15:25.523115 (+ 315us) client_negotiation.cc:240] Negotiation successful
0925 05:15:25.523132 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260925 05:15:25.523267 18707 raft_consensus.cc:2454] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20d65887628e4bd1ac35dd2c794bb494 in term 0.
I20260925 05:15:25.523505 18771 leader_election.cc:302] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20d65887628e4bd1ac35dd2c794bb494, d0df1ee0f6344fa398d26c764eca485d; no voters:
I20260925 05:15:25.523509 18969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a147d774042e58f8ecb88db3da755" candidate_uuid: "20d65887628e4bd1ac35dd2c794bb494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" is_pre_election: true
I20260925 05:15:25.523654 18969 raft_consensus.cc:2454] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20d65887628e4bd1ac35dd2c794bb494 in term 0.
I20260925 05:15:25.523636 19036 raft_consensus.cc:2781] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:25.523695 19036 raft_consensus.cc:482] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:25.523741 19036 raft_consensus.cc:3037] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:25.524448 19036 raft_consensus.cc:504] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.524593 19036 leader_election.cc:288] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [CANDIDATE]: Term 1 election: Requested vote from peers d0df1ee0f6344fa398d26c764eca485d (127.13.160.129:36315), 15c41a63a3a141f4be4136b6cfce9cf4 (127.13.160.131:40263)
I20260925 05:15:25.524756 18707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a147d774042e58f8ecb88db3da755" candidate_uuid: "20d65887628e4bd1ac35dd2c794bb494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0df1ee0f6344fa398d26c764eca485d"
I20260925 05:15:25.524802 18969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a147d774042e58f8ecb88db3da755" candidate_uuid: "20d65887628e4bd1ac35dd2c794bb494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c41a63a3a141f4be4136b6cfce9cf4"
I20260925 05:15:25.524857 18707 raft_consensus.cc:3037] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:25.524858 18969 raft_consensus.cc:3037] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:25.525583 18969 raft_consensus.cc:2454] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20d65887628e4bd1ac35dd2c794bb494 in term 1.
I20260925 05:15:25.525703 18707 raft_consensus.cc:2454] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20d65887628e4bd1ac35dd2c794bb494 in term 1.
I20260925 05:15:25.525748 18771 leader_election.cc:302] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c41a63a3a141f4be4136b6cfce9cf4, 20d65887628e4bd1ac35dd2c794bb494; no voters:
I20260925 05:15:25.525857 19036 raft_consensus.cc:2781] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:25.525995 19036 raft_consensus.cc:686] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 1 LEADER]: Becoming Leader. State: Replica: 20d65887628e4bd1ac35dd2c794bb494, State: Running, Role: LEADER
I20260925 05:15:25.526126 19036 consensus_queue.cc:237] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } }
I20260925 05:15:25.527033 18563 catalog_manager.cc:5495] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20d65887628e4bd1ac35dd2c794bb494 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "20d65887628e4bd1ac35dd2c794bb494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20d65887628e4bd1ac35dd2c794bb494" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35595 } health_report { overall_health: HEALTHY } } }
I20260925 05:15:25.543305 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.542241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.13.160.130:35595)
0925 05:15:25.542332 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.542334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.542344 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:25.542396 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.542397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.542410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.542471 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.543063 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.543064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.543064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.543068 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.543082 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.543086 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.543141 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.543180 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.543192 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 05:15:25.545189 18707 raft_consensus.cc:1261] T 494a147d774042e58f8ecb88db3da755 P d0df1ee0f6344fa398d26c764eca485d [term 1 FOLLOWER]: Refusing update from remote peer 20d65887628e4bd1ac35dd2c794bb494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:25.545251 18969 raft_consensus.cc:1261] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 1 FOLLOWER]: Refusing update from remote peer 20d65887628e4bd1ac35dd2c794bb494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:25.545549 19036 consensus_queue.cc:1035] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0df1ee0f6344fa398d26c764eca485d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:25.545723 19039 consensus_queue.cc:1035] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c41a63a3a141f4be4136b6cfce9cf4" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:25.549638 19044 mvcc.cc:205] Tried to move back new op lower bound from 7333123381430120448 to 7333123381355552768. Current Snapshot: MvccSnapshot[applied={T|T < 7333123381430120448 or (T in {7333123381430120448})}]
I20260925 05:15:25.550621 18565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:25.550738 18565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:25.551625 18565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a14a545633a42749fd8a672fa26289b: Sending DeleteTablet for 3 replicas of tablet 494a147d774042e58f8ecb88db3da755
I20260925 05:15:25.551972 18687 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a147d774042e58f8ecb88db3da755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:25 UTC) from {username='slave'} at 127.0.0.1:57744
I20260925 05:15:25.552022 18949 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a147d774042e58f8ecb88db3da755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:25 UTC) from {username='slave'} at 127.0.0.1:49688
I20260925 05:15:25.552045 18818 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a147d774042e58f8ecb88db3da755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:25 UTC) from {username='slave'} at 127.0.0.1:43310
I20260925 05:15:25.552265 19054 tablet_replica.cc:330] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: stopping tablet replica
I20260925 05:15:25.552268 19053 tablet_replica.cc:330] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: stopping tablet replica
I20260925 05:15:25.552366 19054 raft_consensus.cc:2229] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:25.552465 19054 raft_consensus.cc:2258] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:25.552477 19053 raft_consensus.cc:2229] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:25.552626 19053 raft_consensus.cc:2258] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:25.552690 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18625
I20260925 05:15:25.553400 19054 ts_tablet_manager.cc:1905] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:25.553568 19053 ts_tablet_manager.cc:1905] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:25.555455 19053 ts_tablet_manager.cc:1918] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:25.555527 19053 log.cc:1196] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/494a147d774042e58f8ecb88db3da755
I20260925 05:15:25.555523 19054 ts_tablet_manager.cc:1918] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:25.555604 19054 log.cc:1196] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/494a147d774042e58f8ecb88db3da755
I20260925 05:15:25.555858 19053 ts_tablet_manager.cc:1939] T 494a147d774042e58f8ecb88db3da755 P 15c41a63a3a141f4be4136b6cfce9cf4: Deleting consensus metadata
I20260925 05:15:25.555891 19054 ts_tablet_manager.cc:1939] T 494a147d774042e58f8ecb88db3da755 P 20d65887628e4bd1ac35dd2c794bb494: Deleting consensus metadata
I20260925 05:15:25.557001 18549 catalog_manager.cc:4841] TS 20d65887628e4bd1ac35dd2c794bb494 (127.13.160.130:35595): tablet 494a147d774042e58f8ecb88db3da755 (table test-table [id=2b1cdc81244a4353a54410d3398ec70b]) successfully deleted
I20260925 05:15:25.557130 18549 catalog_manager.cc:4841] TS 15c41a63a3a141f4be4136b6cfce9cf4 (127.13.160.131:40263): tablet 494a147d774042e58f8ecb88db3da755 (table test-table [id=2b1cdc81244a4353a54410d3398ec70b]) successfully deleted
W20260925 05:15:25.560393 18549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:36315 (error 108)
I20260925 05:15:25.560575 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18756
I20260925 05:15:25.562517 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.562356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:36315 (local address 127.0.0.1:57752)
0925 05:15:25.562429 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.562483 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:36315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 05:15:25.562680 18549 catalog_manager.cc:4584] TS d0df1ee0f6344fa398d26c764eca485d (127.13.160.129:36315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 494a147d774042e58f8ecb88db3da755: Network error: Client connection negotiation failed: client connection to 127.13.160.129:36315: connect: Connection refused (error 111)
I20260925 05:15:25.565979 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18887
I20260925 05:15:25.570561 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 18534
2026-09-25T05:15:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (742 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 05:15:25.590945 13954 mini_oidc.cc:152] Starting JWKS server
I20260925 05:15:25.592118 13954 webserver.cc:460] Webserver started at https://127.0.0.1:34519/ using document root <none> and password file <none>
I20260925 05:15:25.592249 13954 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 05:15:25.592444 13954 webserver.cc:460] Webserver started at http://127.0.0.1:39219/ using document root <none> and password file <none>
2026-09-25T05:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:25Z Disabled control of system clock
I20260925 05:15:25.600764 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45843
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:45843
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/testchainca.cert with env {}
W20260925 05:15:25.684854 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:25.685101 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:25.685127 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:25.686364 19061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:25.686414 19061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:25.686426 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:25.686439 19061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:25.686451 19061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 05:15:25.686465 19061 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 05:15:25.686475 19061 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34519/jwks.json
I20260925 05:15:25.688130 19061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:45843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:25.688366 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:25.688552 19061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:25.693825 19066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.693836 19067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.693837 19069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.694147 19061 server_base.cc:1034] running on GCE node
I20260925 05:15:25.694439 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:25.694808 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:25.696017 19061 hybrid_clock.cc:648] HybridClock initialized: now 1790313325696001 us; error 32 us; skew 500 ppm
I20260925 05:15:25.697407 19061 webserver.cc:460] Webserver started at http://127.13.160.190:45081/ using document root <none> and password file <none>
I20260925 05:15:25.697693 19061 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:25.697773 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:25.697934 19061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:25.699191 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "e42848e13f1345629fb854f8c654bcd5"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "88b0dbaac0c2f61878d35138eb5af425"
server_key_iv: "311bf19f1523e4aca091fe10f3f7d2a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.699586 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "e42848e13f1345629fb854f8c654bcd5"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "88b0dbaac0c2f61878d35138eb5af425"
server_key_iv: "311bf19f1523e4aca091fe10f3f7d2a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.700948 19061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:25.701741 19075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.701924 19061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:25.702045 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "e42848e13f1345629fb854f8c654bcd5"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "88b0dbaac0c2f61878d35138eb5af425"
server_key_iv: "311bf19f1523e4aca091fe10f3f7d2a7"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.702127 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:25.727969 19061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:25.728358 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:25.728539 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:25.732955 19061 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:45843
I20260925 05:15:25.733006 19127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:45843 every 8 connection(s)
I20260925 05:15:25.733377 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:25.734357 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:25.735630 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19061
I20260925 05:15:25.735739 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:25.735918 13954 external_mini_cluster.cc:1428] Setting key a29af180eae8dc3252f97b12c170de0f
I20260925 05:15:25.737320 19128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5: Bootstrap starting.
I20260925 05:15:25.738062 19128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:25.738358 19128 log.cc:824] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:25.739310 19128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5: No bootstrap required, opened a new log
I20260925 05:15:25.741809 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.736517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.13.160.190:45843)
0925 05:15:25.736716 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.736724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.736750 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:15:25.737178 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.737185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.737249 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.737377 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.737619 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.738710 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.739852 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.740277 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.740371 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.741100 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.741104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.741104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.741281 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.741315 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.741326 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.741377 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.741484 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.741549 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260925 05:15:25.741804 19128 raft_consensus.cc:348] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } }
I20260925 05:15:25.741979 19128 raft_consensus.cc:374] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:25.742019 19128 raft_consensus.cc:729] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e42848e13f1345629fb854f8c654bcd5, State: Initialized, Role: FOLLOWER
I20260925 05:15:25.742152 19128 consensus_queue.cc:260] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } }
I20260925 05:15:25.742233 19128 raft_consensus.cc:388] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:25.742264 19128 raft_consensus.cc:482] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:25.742311 19128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:25.742862 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:15:25.743144 19128 raft_consensus.cc:504] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } }
I20260925 05:15:25.743251 19128 leader_election.cc:302] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e42848e13f1345629fb854f8c654bcd5; no voters:
I20260925 05:15:25.743430 19128 leader_election.cc:288] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:25.743494 19133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:25.743711 19133 raft_consensus.cc:686] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [term 1 LEADER]: Becoming Leader. State: Replica: e42848e13f1345629fb854f8c654bcd5, State: Running, Role: LEADER
I20260925 05:15:25.743729 19128 sys_catalog.cc:564] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:25.743814 19133 consensus_queue.cc:237] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } }
I20260925 05:15:25.744153 19133 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e42848e13f1345629fb854f8c654bcd5. Latest consensus state: current_term: 1 leader_uuid: "e42848e13f1345629fb854f8c654bcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } } }
I20260925 05:15:25.744248 19133 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:25.744167 19134 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e42848e13f1345629fb854f8c654bcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42848e13f1345629fb854f8c654bcd5" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45843 } } }
I20260925 05:15:25.744398 19134 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:25.744508 19140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:25.745054 19140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:25.746726 19140 catalog_manager.cc:1347] Generated new cluster ID: 775a56e10bc349bc8b6d864ed6751951
I20260925 05:15:25.746785 19140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:25.758000 19140 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:25.758571 19140 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:25.763751 19140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e42848e13f1345629fb854f8c654bcd5: Generated new TSK 0
I20260925 05:15:25.763999 19140 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 05:15:25.796947 19158 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 05:15:25.797060 19158 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 05:15:25.797224 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.774488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.13.160.190:45843)
0925 05:15:25.774557 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.774560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.774705 (+ 145us) server_negotiation.cc:407] Connection header received
0925 05:15:25.774816 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.774817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.774829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.774915 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0925 05:15:25.775089 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.775596 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.776385 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.776590 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.776681 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.776911 (+ 230us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 05:15:25.796955 (+ 20044us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 05:15:25.797150 (+ 195us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 05:15:25.797297 19131 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 05:15:25.797698 19157 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:25.774427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45843 (local address 127.0.0.1:49998)
0925 05:15:25.774633 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.774649 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.774725 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.774926 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.774928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.774930 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 05:15:25.775006 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:25.775009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.775690 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.775691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:25.776295 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:25.776297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.776692 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.776693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:25.776756 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.776784 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 05:15:25.797180 (+ 20396us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 05:15:25.797407 (+ 227us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.160.190:45843: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.160.190:45843: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 05:15:25.798182 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19061
2026-09-25T05:15:25Z chronyd exiting
I20260925 05:15:25.834074 13954 test_util.cc:175] -----------------------------------------------
I20260925 05:15:25.834165 13954 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790313300911901-13954-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 05:15:25.835026 13954 mini_oidc.cc:152] Starting JWKS server
I20260925 05:15:25.835739 13954 webserver.cc:460] Webserver started at https://127.0.0.1:39155/ using document root <none> and password file <none>
I20260925 05:15:25.835808 13954 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 05:15:25.835951 13954 webserver.cc:460] Webserver started at http://127.0.0.1:32837/ using document root <none> and password file <none>
2026-09-25T05:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:25Z Disabled control of system clock
I20260925 05:15:25.843607 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38225
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:38225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39155/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/testchainca.cert with env {}
W20260925 05:15:25.924127 19164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:25.924352 19164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:25.924376 19164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:25.925870 19164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:25.925927 19164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:25.925951 19164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:25.925966 19164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:25.925978 19164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 05:15:25.926031 19164 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 05:15:25.926066 19164 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39155/jwks.json
I20260925 05:15:25.927870 19164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:38225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39155/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:25.928186 19164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:25.928438 19164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:25.933508 19172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:25.933509 19170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.933539 19164 server_base.cc:1034] running on GCE node
W20260925 05:15:25.933522 19169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:25.933970 19164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:25.934300 19164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:25.935443 19164 hybrid_clock.cc:648] HybridClock initialized: now 1790313325935417 us; error 37 us; skew 500 ppm
I20260925 05:15:25.936864 19164 webserver.cc:460] Webserver started at http://127.13.160.190:46071/ using document root <none> and password file <none>
I20260925 05:15:25.937135 19164 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:25.937183 19164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:25.937358 19164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:25.938457 19164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "de11a5f05f9642148211c8c50ec5b607"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "a5c8b9156988ad33a36f351396f95650"
server_key_iv: "e825f8bdda0e13ee13c134705ca7e422"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.938834 19164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "de11a5f05f9642148211c8c50ec5b607"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "a5c8b9156988ad33a36f351396f95650"
server_key_iv: "e825f8bdda0e13ee13c134705ca7e422"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.940261 19164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:25.941154 19178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:25.941360 19164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:25.941468 19164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "de11a5f05f9642148211c8c50ec5b607"
format_stamp: "Formatted at 2026-09-25 05:15:25 on dist-test-slave-9tgz"
server_key: "a5c8b9156988ad33a36f351396f95650"
server_key_iv: "e825f8bdda0e13ee13c134705ca7e422"
server_key_version: "encryptionkey@0"
I20260925 05:15:25.941555 19164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:25.953492 19164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:25.953857 19164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:25.954010 19164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:25.958480 19164 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:38225
I20260925 05:15:25.958518 19230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:38225 every 8 connection(s)
I20260925 05:15:25.958932 19164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:25.959611 19231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:25.962395 19231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607: Bootstrap starting.
I20260925 05:15:25.963155 19231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:25.963449 19231 log.cc:824] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:25.964160 19231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607: No bootstrap required, opened a new log
I20260925 05:15:25.965833 19231 raft_consensus.cc:348] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } }
I20260925 05:15:25.965981 19231 raft_consensus.cc:374] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:25.966004 19231 raft_consensus.cc:729] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11a5f05f9642148211c8c50ec5b607, State: Initialized, Role: FOLLOWER
I20260925 05:15:25.966173 19231 consensus_queue.cc:260] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } }
I20260925 05:15:25.966236 19231 raft_consensus.cc:388] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:25.966259 19231 raft_consensus.cc:482] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:25.966323 19231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:25.966977 19231 raft_consensus.cc:504] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } }
I20260925 05:15:25.967087 19231 leader_election.cc:302] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de11a5f05f9642148211c8c50ec5b607; no voters:
I20260925 05:15:25.967278 19231 leader_election.cc:288] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:25.967324 19234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:25.967518 19234 raft_consensus.cc:686] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [term 1 LEADER]: Becoming Leader. State: Replica: de11a5f05f9642148211c8c50ec5b607, State: Running, Role: LEADER
I20260925 05:15:25.967556 19231 sys_catalog.cc:564] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:25.967624 19234 consensus_queue.cc:237] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } }
I20260925 05:15:25.967875 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19164
I20260925 05:15:25.967967 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:25.967981 19236 sys_catalog.cc:455] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de11a5f05f9642148211c8c50ec5b607. Latest consensus state: current_term: 1 leader_uuid: "de11a5f05f9642148211c8c50ec5b607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } } }
I20260925 05:15:25.968077 19236 sys_catalog.cc:458] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:25.968166 13954 external_mini_cluster.cc:1428] Setting key 8fe2933f43a2871989451f39bcd37c7a
I20260925 05:15:25.968641 19235 sys_catalog.cc:455] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de11a5f05f9642148211c8c50ec5b607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de11a5f05f9642148211c8c50ec5b607" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38225 } } }
I20260925 05:15:25.968796 19235 sys_catalog.cc:458] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:25.968858 19241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:25.969347 19241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:25.971287 19241 catalog_manager.cc:1347] Generated new cluster ID: f98b9905e1144e228f06a79c5d771724
I20260925 05:15:25.971341 19241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:25.975502 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.968778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.13.160.190:38225)
0925 05:15:25.969698 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.969707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.969969 (+ 262us) server_negotiation.cc:407] Connection header received
0925 05:15:25.970200 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.970210 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.970291 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.970451 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:25.970639 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.972057 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.973162 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.973784 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.974219 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.974671 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:25.974675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:25.974675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:25.974879 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:25.974951 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:25.974965 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:25.975174 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:25.975205 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 05:15:25.975278 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":70,"threads_started":1}
I20260925 05:15:25.976732 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:15:25.977980 19241 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:25.978636 19241 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:25.983067 19241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de11a5f05f9642148211c8c50ec5b607: Generated new TSK 0
I20260925 05:15:25.983286 19241 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 05:15:26.011595 19261 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 05:15:26.011694 19261 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 05:15:26.011991 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:25.989294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.13.160.190:38225)
0925 05:15:25.989361 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:25.989363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:25.989497 (+ 134us) server_negotiation.cc:407] Connection header received
0925 05:15:25.989581 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:25.989582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:25.989594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:25.989679 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0925 05:15:25.989824 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.990337 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.991237 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.991442 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.991517 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.991707 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 05:15:26.011751 (+ 20044us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39155/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 05:15:26.011935 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39155/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 05:15:26.012077 19246 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41646: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39155/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 05:15:26.012106 19260 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:25.989215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38225 (local address 127.0.0.1:41646)
0925 05:15:25.989421 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:25.989430 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:25.989509 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:25.989676 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:25.989677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:25.989679 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 05:15:25.989744 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:25.989746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.990443 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.990444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:25.991140 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:25.991143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:25.991516 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:25.991517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:25.991590 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:25.991595 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 05:15:26.011887 (+ 20292us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39155/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 05:15:26.012005 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.160.190:38225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39155/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":113,"threads_started":1}
I20260925 05:15:26.012462 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19164
2026-09-25T05:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (215 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 05:15:26.050374 13954 mini_oidc.cc:152] Starting JWKS server
I20260925 05:15:26.051275 13954 webserver.cc:460] Webserver started at https://127.0.0.1:34885/ using document root <none> and password file <none>
I20260925 05:15:26.051324 13954 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 05:15:26.051496 13954 webserver.cc:460] Webserver started at http://127.0.0.1:40631/ using document root <none> and password file <none>
2026-09-25T05:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:26Z Disabled control of system clock
I20260925 05:15:26.059933 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:42809
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:44235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:42809
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34885/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 05:15:26.143021 19267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:26.143251 19267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:26.143275 19267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:26.144522 19267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:26.144568 19267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:26.144580 19267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:26.144593 19267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:26.144606 19267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 05:15:26.144619 19267 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 05:15:26.144630 19267 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34885/jwks.json
I20260925 05:15:26.146014 19267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:42809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34885/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:26.146224 19267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:26.146476 19267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:26.151250 19275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:26.151355 19267 server_base.cc:1034] running on GCE node
W20260925 05:15:26.151260 19273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:26.151263 19272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:26.151703 19267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:26.152065 19267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:26.153209 19267 hybrid_clock.cc:648] HybridClock initialized: now 1790313326153190 us; error 38 us; skew 500 ppm
I20260925 05:15:26.154479 19267 webserver.cc:460] Webserver started at http://127.13.160.190:37489/ using document root <none> and password file <none>
I20260925 05:15:26.154726 19267 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:26.154771 19267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:26.154944 19267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:26.156046 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "802cea59d79e4429b22e14d38658c2b3"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "3d167558aed71af22818daaf10744e08"
server_key_iv: "4f52aede20c5a647aba17d9e7495e07c"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.156409 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "802cea59d79e4429b22e14d38658c2b3"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "3d167558aed71af22818daaf10744e08"
server_key_iv: "4f52aede20c5a647aba17d9e7495e07c"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.157815 19267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:26.158548 19281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:26.158732 19267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:26.158834 19267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "802cea59d79e4429b22e14d38658c2b3"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "3d167558aed71af22818daaf10744e08"
server_key_iv: "4f52aede20c5a647aba17d9e7495e07c"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.158946 19267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:26.188722 19267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:26.189134 19267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:26.189285 19267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:26.193300 19267 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:42809
I20260925 05:15:26.193334 19333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:42809 every 8 connection(s)
I20260925 05:15:26.193655 19267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:26.194315 19334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:26.194721 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19267
I20260925 05:15:26.194825 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:26.195067 13954 external_mini_cluster.cc:1428] Setting key 173c5f7284fd30d80232f0853a5e6422
I20260925 05:15:26.198110 19334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3: Bootstrap starting.
I20260925 05:15:26.198833 19334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:26.199185 19334 log.cc:824] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:26.200156 19334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3: No bootstrap required, opened a new log
I20260925 05:15:26.200901 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:26.195676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.13.160.190:42809)
0925 05:15:26.195860 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:26.195868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:26.195897 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:15:26.196460 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:26.196467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:26.196550 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:26.196669 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:26.196910 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.197792 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.198860 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.199352 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.199457 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.200207 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:26.200212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:26.200212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:26.200411 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:26.200442 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:26.200453 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:26.200558 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:26.200639 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 05:15:26.200725 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260925 05:15:26.201802 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:15:26.202409 19334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } }
I20260925 05:15:26.202589 19334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:26.202625 19334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802cea59d79e4429b22e14d38658c2b3, State: Initialized, Role: FOLLOWER
I20260925 05:15:26.202749 19334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } }
I20260925 05:15:26.202833 19334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:26.202857 19334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:26.202934 19334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:26.203472 19334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } }
I20260925 05:15:26.203554 19334 leader_election.cc:302] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 802cea59d79e4429b22e14d38658c2b3; no voters:
I20260925 05:15:26.203681 19334 leader_election.cc:288] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:26.203759 19339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:26.203933 19334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:26.203948 19339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [term 1 LEADER]: Becoming Leader. State: Replica: 802cea59d79e4429b22e14d38658c2b3, State: Running, Role: LEADER
I20260925 05:15:26.204087 19339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } }
I20260925 05:15:26.204494 19341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802cea59d79e4429b22e14d38658c2b3. Latest consensus state: current_term: 1 leader_uuid: "802cea59d79e4429b22e14d38658c2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } } }
I20260925 05:15:26.204502 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "802cea59d79e4429b22e14d38658c2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802cea59d79e4429b22e14d38658c2b3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42809 } } }
I20260925 05:15:26.204660 19341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:26.205015 19348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:26.204686 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:26.205550 19348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:26.207086 19348 catalog_manager.cc:1347] Generated new cluster ID: 1a14539847ec432a8d9a7dece5d96fae
I20260925 05:15:26.207159 19348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:26.224380 19348 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:26.224964 19348 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:26.232997 19348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 802cea59d79e4429b22e14d38658c2b3: Generated new TSK 0
I20260925 05:15:26.233242 19348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:26.243026 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:26.239687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.13.160.190:42809)
0925 05:15:26.239730 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:26.239732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:26.239853 (+ 121us) server_negotiation.cc:407] Connection header received
0925 05:15:26.239953 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:26.239953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:26.239964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:26.240040 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0925 05:15:26.240170 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.240678 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.241459 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.241667 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.241741 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.241945 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 05:15:26.242770 (+ 825us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 05:15:26.242948 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51782: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 05:15:26.243094 19363 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:26.239624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:42809 (local address 127.0.0.1:51782)
0925 05:15:26.239775 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:26.239785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:26.239872 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:26.240037 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:26.240038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:26.240040 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 05:15:26.240103 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:26.240105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.240769 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.240770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:26.241372 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:26.241373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.241758 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.241759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:26.241817 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.241822 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 05:15:26.242874 (+ 1052us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 05:15:26.242993 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.160.190:42809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260925 05:15:26.243110 19337 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51782: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260925 05:15:26.243320 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19267
E20260925 05:15:26.254312 19364 webserver.cc:533] Webserver: sslize failed:
E20260925 05:15:26.254408 19364 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T05:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T05:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:26Z Disabled control of system clock
I20260925 05:15:26.289024 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40609
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40609
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/testchainca.cert with env {}
W20260925 05:15:26.371886 19368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:26.372229 19368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:26.372269 19368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:26.373534 19368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:26.373611 19368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:26.373631 19368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:26.373653 19368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:26.373684 19368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 05:15:26.373708 19368 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 05:15:26.373735 19368 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 05:15:26.375397 19368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:26.375734 19368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:26.375983 19368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:26.381184 19374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:26.381268 19368 server_base.cc:1034] running on GCE node
W20260925 05:15:26.381237 19376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:26.381183 19373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:26.381672 19368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:26.382019 19368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:26.383183 19368 hybrid_clock.cc:648] HybridClock initialized: now 1790313326383170 us; error 47 us; skew 500 ppm
I20260925 05:15:26.384467 19368 webserver.cc:460] Webserver started at http://127.13.160.190:46225/ using document root <none> and password file <none>
I20260925 05:15:26.384737 19368 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:26.384809 19368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:26.384959 19368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:26.386104 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "b043ad2a9d9c4c419e84ace03114395a"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "4ff2d4021605902e6e7e657a572ee94c"
server_key_iv: "05d39598eca25b8bade9b454f9249815"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.386493 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "b043ad2a9d9c4c419e84ace03114395a"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "4ff2d4021605902e6e7e657a572ee94c"
server_key_iv: "05d39598eca25b8bade9b454f9249815"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.388006 19368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:26.388823 19382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:26.389017 19368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:26.389128 19368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "b043ad2a9d9c4c419e84ace03114395a"
format_stamp: "Formatted at 2026-09-25 05:15:26 on dist-test-slave-9tgz"
server_key: "4ff2d4021605902e6e7e657a572ee94c"
server_key_iv: "05d39598eca25b8bade9b454f9249815"
server_key_version: "encryptionkey@0"
I20260925 05:15:26.389209 19368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:26.414996 19368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:26.415366 19368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:26.415529 19368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:26.419591 19368 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40609
I20260925 05:15:26.419631 19434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40609 every 8 connection(s)
I20260925 05:15:26.420024 19368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:26.421087 19435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:26.423499 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a: Bootstrap starting.
I20260925 05:15:26.423561 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19368
I20260925 05:15:26.423694 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:26.423867 13954 external_mini_cluster.cc:1428] Setting key 65d8fe283c2fba0444544f507d04c366
I20260925 05:15:26.424319 19435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:26.424714 19435 log.cc:824] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:26.425534 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a: No bootstrap required, opened a new log
I20260925 05:15:26.427732 19435 raft_consensus.cc:348] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } }
I20260925 05:15:26.427937 19435 raft_consensus.cc:374] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:26.427966 19435 raft_consensus.cc:729] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b043ad2a9d9c4c419e84ace03114395a, State: Initialized, Role: FOLLOWER
I20260925 05:15:26.428126 19435 consensus_queue.cc:260] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } }
I20260925 05:15:26.428217 19435 raft_consensus.cc:388] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:26.428260 19435 raft_consensus.cc:482] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:26.428311 19435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:26.429196 19435 raft_consensus.cc:504] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } }
I20260925 05:15:26.429306 19435 leader_election.cc:302] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b043ad2a9d9c4c419e84ace03114395a; no voters:
I20260925 05:15:26.429512 19435 leader_election.cc:288] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:26.429540 19440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:26.429646 19440 raft_consensus.cc:686] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [term 1 LEADER]: Becoming Leader. State: Replica: b043ad2a9d9c4c419e84ace03114395a, State: Running, Role: LEADER
I20260925 05:15:26.429721 19440 consensus_queue.cc:237] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } }
I20260925 05:15:26.429792 19435 sys_catalog.cc:564] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:26.429982 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:26.424424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.13.160.190:40609)
0925 05:15:26.424650 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:26.424658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:26.424684 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:15:26.425316 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:26.425324 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:26.425391 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:26.425521 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:26.425770 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.426656 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.427828 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.428429 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.428525 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.429195 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:26.429199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:26.429199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:26.429457 (+ 258us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:26.429500 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:26.429511 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:26.429582 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:26.429663 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 05:15:26.429747 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260925 05:15:26.430466 19440 sys_catalog.cc:455] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b043ad2a9d9c4c419e84ace03114395a. Latest consensus state: current_term: 1 leader_uuid: "b043ad2a9d9c4c419e84ace03114395a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } } }
I20260925 05:15:26.430550 19440 sys_catalog.cc:458] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:26.430704 19441 sys_catalog.cc:455] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b043ad2a9d9c4c419e84ace03114395a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b043ad2a9d9c4c419e84ace03114395a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40609 } } }
I20260925 05:15:26.430868 19441 sys_catalog.cc:458] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:26.430770 19449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:26.431152 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:15:26.431459 19449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:26.433256 19449 catalog_manager.cc:1347] Generated new cluster ID: 909ddd46db7449159180f3b5d17408c8
I20260925 05:15:26.433318 19449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:26.446281 19449 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:26.446867 19449 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:26.452347 19449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b043ad2a9d9c4c419e84ace03114395a: Generated new TSK 0
I20260925 05:15:26.452576 19449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:26.563409 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:26.464093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.13.160.190:40609)
0925 05:15:26.464163 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:26.464165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:26.464258 (+ 93us) server_negotiation.cc:407] Connection header received
0925 05:15:26.464318 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:26.464319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:26.464332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:26.464417 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0925 05:15:26.464528 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.465394 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.466183 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.466509 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.466579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.466751 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 05:15:26.563185 (+ 96434us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 05:15:26.563347 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59090: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 05:15:26.563514 19438 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59090: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 05:15:26.563565 19464 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:26.464008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40609 (local address 127.0.0.1:59090)
0925 05:15:26.464183 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:26.464202 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:26.464256 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:26.464422 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:26.464423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:26.464425 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 05:15:26.464475 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:26.464476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.465476 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.465477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:26.466098 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:26.466100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:26.466603 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:26.466604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:26.466644 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:26.466649 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 05:15:26.563335 (+ 96686us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 05:15:26.563463 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.160.190:40609: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260925 05:15:26.563959 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19368
2026-09-25T05:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (300 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:26 dist-test-slave-9tgz krb5kdc[19476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:26 dist-test-slave-9tgz krb5kdc[19476](info): set up 2 sockets
Sep 25 05:15:26 dist-test-slave-9tgz krb5kdc[19476](info): commencing operation
krb5kdc: starting...
W20260925 05:15:27.775961 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:27 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:27.894017 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38171
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:38171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:27.977661 19489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:27.977909 19489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:27.977934 19489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:27.979319 19489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:27.979374 19489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:27.979390 19489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:27.979403 19489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:27.979416 19489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:27.981058 19489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:38171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:27.981329 19489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:27.981596 19489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:27.986526 19494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:27.986575 19495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:27.986685 19489 server_base.cc:1034] running on GCE node
W20260925 05:15:27.986526 19497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:27.987085 19489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:27.987416 19489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:27.988576 19489 hybrid_clock.cc:648] HybridClock initialized: now 1790313327988561 us; error 27 us; skew 500 ppm
Sep 25 05:15:27 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313327, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:27.991485 19489 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:27.992023 19489 webserver.cc:460] Webserver started at http://127.13.160.190:37549/ using document root <none> and password file <none>
I20260925 05:15:27.992290 19489 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:27.992338 19489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:27.992496 19489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:27.993659 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f"
format_stamp: "Formatted at 2026-09-25 05:15:27 on dist-test-slave-9tgz"
server_key: "dc78bc6925235565edb34bb363613554"
server_key_iv: "a6667c46bbd75063c245780903452fbc"
server_key_version: "encryptionkey@0"
I20260925 05:15:27.994053 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f"
format_stamp: "Formatted at 2026-09-25 05:15:27 on dist-test-slave-9tgz"
server_key: "dc78bc6925235565edb34bb363613554"
server_key_iv: "a6667c46bbd75063c245780903452fbc"
server_key_version: "encryptionkey@0"
I20260925 05:15:27.995524 19489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:27.996337 19504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:27.996539 19489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:27.996644 19489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f"
format_stamp: "Formatted at 2026-09-25 05:15:27 on dist-test-slave-9tgz"
server_key: "dc78bc6925235565edb34bb363613554"
server_key_iv: "a6667c46bbd75063c245780903452fbc"
server_key_version: "encryptionkey@0"
I20260925 05:15:27.996727 19489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:28.009721 19489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:28.012228 19489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:28.012347 19489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:28.016183 19489 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:38171
I20260925 05:15:28.016253 19556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:38171 every 8 connection(s)
I20260925 05:15:28.016593 19489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:28.017539 19557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:28.019061 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19489
I20260925 05:15:28.019152 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:28.019337 13954 external_mini_cluster.cc:1428] Setting key f65296430f097f4fc7996199494b1f7e
I20260925 05:15:28.020449 19557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f: Bootstrap starting.
I20260925 05:15:28.021222 19557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:28.021569 19557 log.cc:824] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:28.022318 19557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f: No bootstrap required, opened a new log
I20260925 05:15:28.024806 19557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } }
I20260925 05:15:28.025022 19557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:28.025066 19557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad7cf1c0aa7473fb0a4292b6e0ee19f, State: Initialized, Role: FOLLOWER
I20260925 05:15:28.025257 19557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } }
I20260925 05:15:28.025382 19557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:28.025419 19557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:28.025475 19557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:28.026367 19557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } }
I20260925 05:15:28.026502 19557 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad7cf1c0aa7473fb0a4292b6e0ee19f; no voters:
I20260925 05:15:28.026715 19557 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:28.026739 19562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 1 FOLLOWER]: Leader election won for term 1
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:28.026858 19562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [term 1 LEADER]: Becoming Leader. State: Replica: 2ad7cf1c0aa7473fb0a4292b6e0ee19f, State: Running, Role: LEADER
I20260925 05:15:28.026974 19562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } }
I20260925 05:15:28.027042 19557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:28.027333 19563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } } }
I20260925 05:15:28.027432 19563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:28.027319 19562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad7cf1c0aa7473fb0a4292b6e0ee19f. Latest consensus state: current_term: 1 leader_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad7cf1c0aa7473fb0a4292b6e0ee19f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38171 } } }
I20260925 05:15:28.027642 19562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:28.027781 19566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:28.028522 19566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:28.031019 19566 catalog_manager.cc:1347] Generated new cluster ID: e63eed4d213f445d86446088652ae45b
I20260925 05:15:28.031081 19566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:28.031939 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.019934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.13.160.190:38171)
0925 05:15:28.020114 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.020122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.020147 (+ 25us) server_negotiation.cc:407] Connection header received
0925 05:15:28.020631 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.020639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.020694 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.020818 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.021701 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.022713 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.023825 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.024271 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.024372 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.027567 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.027572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.027572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.027587 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.030079 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.030509 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.030513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.030514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.030623 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.030864 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.030865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.030865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.031232 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.031399 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.031565 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.031713 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260925 05:15:28.043237 19566 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:28.043846 19566 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:28.047899 19566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad7cf1c0aa7473fb0a4292b6e0ee19f: Generated new TSK 0
I20260925 05:15:28.048137 19566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:28.081506 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:28.167441 19583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:28.167673 19583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:28.167699 19583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:28.168944 19583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:28.169008 19583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:28.169050 19583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:28.170646 19583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:28.170861 19583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:28.171142 19583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:28.176321 19591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:28.176457 19583 server_base.cc:1034] running on GCE node
W20260925 05:15:28.176321 19589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:28.176406 19588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:28.176753 19583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:28.177011 19583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:28.178184 19583 hybrid_clock.cc:648] HybridClock initialized: now 1790313328178158 us; error 41 us; skew 500 ppm
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.180868 19583 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:28.181420 19583 webserver.cc:460] Webserver started at http://127.13.160.129:43379/ using document root <none> and password file <none>
I20260925 05:15:28.181679 19583 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:28.181753 19583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:28.181910 19583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:28.183225 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "3f2e5b315f1d44a49bde89ffa558eb42"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "19c61f4d750a305d452393b19f475ed3"
server_key_iv: "a4d3c3fda3083f10a4ef8ec1ee481fbd"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.183640 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "3f2e5b315f1d44a49bde89ffa558eb42"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "19c61f4d750a305d452393b19f475ed3"
server_key_iv: "a4d3c3fda3083f10a4ef8ec1ee481fbd"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.185142 19583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:28.185974 19598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.186159 19583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:28.186275 19583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "3f2e5b315f1d44a49bde89ffa558eb42"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "19c61f4d750a305d452393b19f475ed3"
server_key_iv: "a4d3c3fda3083f10a4ef8ec1ee481fbd"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.186371 19583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:28.203336 19583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:28.205919 19583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:28.206051 19583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:28.206262 19583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:28.206619 19583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:28.206657 19583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.206682 19583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:28.206725 19583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.212965 19583 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:37217
I20260925 05:15:28.213008 19711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:37217 every 8 connection(s)
I20260925 05:15:28.213352 19583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:28.216943 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19583
I20260925 05:15:28.217072 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:28.217271 13954 external_mini_cluster.cc:1428] Setting key 33ec35675f201a776f09b99bb56d74f9
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:28.224145 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.214229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:46907 (local address 127.13.160.190:38171)
0925 05:15:28.214288 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.214290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.214466 (+ 176us) server_negotiation.cc:407] Connection header received
0925 05:15:28.215541 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.215542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.215552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.215621 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.216271 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.216828 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.218146 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.218479 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.218558 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.221091 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.221097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.221097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.221107 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.222572 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.223110 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.223111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.223111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.223176 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.223447 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.223448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.223448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.223572 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.223651 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.223953 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.224050 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 05:15:28.224361 19714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.214180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38171 (local address 127.13.160.129:46907)
0925 05:15:28.214342 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:28.214388 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:28.215442 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:28.215703 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:28.215707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:28.215976 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:28.216192 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.216196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.216927 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.216928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.218058 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.218061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.218568 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.218569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.218677 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.219300 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:28.219312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:28.220976 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:28.222689 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.222698 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.222701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.223021 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.223251 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.223252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.223252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.223374 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.223653 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:28.223661 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:28.223816 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 05:15:28.223921 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 05:15:28.224095 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260925 05:15:28.225085 19712 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38171
I20260925 05:15:28.225235 19712 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
I20260925 05:15:28.225642 19712 heartbeater.cc:511] Master 127.13.160.190:38171 requested a full tablet report, sending...
I20260925 05:15:28.226292 19521 ts_manager.cc:194] Registered new tserver with Master: 3f2e5b315f1d44a49bde89ffa558eb42 (127.13.160.129:37217)
I20260925 05:15:28.227129 19521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:46907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:28.245946 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:28.328933 19717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:28.329172 19717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:28.329200 19717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:28.330467 19717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:28.330524 19717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:28.330554 19717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:28.332267 19717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:28.332551 19717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:28.332834 19717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:28.338053 19723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:28.338078 19725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:28.338063 19722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:28.338342 19717 server_base.cc:1034] running on GCE node
I20260925 05:15:28.338551 19717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:28.338766 19717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:28.339911 19717 hybrid_clock.cc:648] HybridClock initialized: now 1790313328339898 us; error 27 us; skew 500 ppm
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.342618 19717 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:28.343190 19717 webserver.cc:460] Webserver started at http://127.13.160.130:37079/ using document root <none> and password file <none>
I20260925 05:15:28.343430 19717 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:28.343513 19717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:28.343688 19717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:28.344864 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "3929ce42bdc64bf7ba85de781edcdf25"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "6a159c9689d9acc008de90dadf29799f"
server_key_iv: "bc8a5739cb85bd5489d7fe7beb2a4f80"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.345269 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "3929ce42bdc64bf7ba85de781edcdf25"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "6a159c9689d9acc008de90dadf29799f"
server_key_iv: "bc8a5739cb85bd5489d7fe7beb2a4f80"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.346693 19717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:28.347618 19732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.347817 19717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:28.347918 19717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "3929ce42bdc64bf7ba85de781edcdf25"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "6a159c9689d9acc008de90dadf29799f"
server_key_iv: "bc8a5739cb85bd5489d7fe7beb2a4f80"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.348004 19717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:28.362562 19717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:28.365255 19717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:28.365438 19717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:28.365710 19717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:28.366047 19717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:28.366102 19717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.366158 19717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:28.366189 19717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.372790 19717 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:39151
I20260925 05:15:28.372869 19845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:39151 every 8 connection(s)
I20260925 05:15:28.373225 19717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:28.381141 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19717
I20260925 05:15:28.381270 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:28.381461 13954 external_mini_cluster.cc:1428] Setting key 403fb6bca3f386ea22f4baf0f50353b5
I20260925 05:15:28.383673 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.374065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:60123 (local address 127.13.160.190:38171)
0925 05:15:28.374144 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.374146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.374360 (+ 214us) server_negotiation.cc:407] Connection header received
0925 05:15:28.375383 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.375385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.375395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.375453 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.376129 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.376613 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.377598 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.377784 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.377833 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.380112 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.380114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.380115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.380125 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.381862 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.382450 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.382451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.382451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.382527 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.382809 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.382810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.382810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.382998 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.383076 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.383412 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.383518 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:28.383692 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.374038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38171 (local address 127.13.160.130:60123)
0925 05:15:28.374232 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:28.374272 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:28.375283 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:28.375551 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:28.375556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:28.375836 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:28.376043 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.376048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.376692 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.376692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.377511 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.377512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.377853 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.377853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.377959 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.378472 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:28.378481 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:28.380001 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:28.381964 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.381974 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.381977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.382350 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.382617 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.382618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.382618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.382739 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.383089 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:28.383095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:28.383272 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 05:15:28.383365 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 05:15:28.383506 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260925 05:15:28.384361 19846 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38171
I20260925 05:15:28.384486 19846 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:28.384706 19846 heartbeater.cc:511] Master 127.13.160.190:38171 requested a full tablet report, sending...
I20260925 05:15:28.385123 19521 ts_manager.cc:194] Registered new tserver with Master: 3929ce42bdc64bf7ba85de781edcdf25 (127.13.160.130:39151)
I20260925 05:15:28.385550 19521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:60123
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:28.410032 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:28.495009 19851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:28.495260 19851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:28.495288 19851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:28.496572 19851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:28.496635 19851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:28.496699 19851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:28.498337 19851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:28.498634 19851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:28.498930 19851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:28.504411 19851 server_base.cc:1034] running on GCE node
W20260925 05:15:28.504391 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:28.504391 19857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:28.504391 19856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:28.504911 19851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:28.505262 19851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:28.506438 19851 hybrid_clock.cc:648] HybridClock initialized: now 1790313328506431 us; error 38 us; skew 500 ppm
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.509104 19851 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:28.509539 19851 webserver.cc:460] Webserver started at http://127.13.160.131:46699/ using document root <none> and password file <none>
I20260925 05:15:28.509744 19851 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:28.509788 19851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:28.509882 19851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:28.511018 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "5cab56c0a2604bcf9fd6a6916c4c3eb6"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "bd977a26dc3807db79deb9f01fd8138a"
server_key_iv: "8ee4278c89b089c5d2406f8438cdd0ed"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.511345 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "5cab56c0a2604bcf9fd6a6916c4c3eb6"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "bd977a26dc3807db79deb9f01fd8138a"
server_key_iv: "8ee4278c89b089c5d2406f8438cdd0ed"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.512704 19851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:28.513458 19866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.513676 19851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:28.513754 19851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "5cab56c0a2604bcf9fd6a6916c4c3eb6"
format_stamp: "Formatted at 2026-09-25 05:15:28 on dist-test-slave-9tgz"
server_key: "bd977a26dc3807db79deb9f01fd8138a"
server_key_iv: "8ee4278c89b089c5d2406f8438cdd0ed"
server_key_version: "encryptionkey@0"
I20260925 05:15:28.513815 19851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:28.523882 19851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:28.526137 19851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:28.526266 19851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:28.526494 19851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:28.526834 19851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:28.526867 19851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.526938 19851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:28.526966 19851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:28.533258 19851 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:38097
I20260925 05:15:28.533295 19979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:38097 every 8 connection(s)
I20260925 05:15:28.533659 19851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:28.535362 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 19851
I20260925 05:15:28.535456 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:28.535653 13954 external_mini_cluster.cc:1428] Setting key 97bd500cf6122df153f493da35f239a0
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:28.543434 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.534461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:57683 (local address 127.13.160.190:38171)
0925 05:15:28.534539 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.534541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.534723 (+ 182us) server_negotiation.cc:407] Connection header received
0925 05:15:28.535920 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.535921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.535936 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.535998 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.536870 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.537363 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.538379 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.538558 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.538615 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.540923 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.540924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.540924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.540931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.542173 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.542553 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.542554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.542554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.542623 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.542838 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.542838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.542839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.543002 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.543056 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.543297 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.543363 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 05:15:28.543672 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.534420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38171 (local address 127.13.160.131:57683)
0925 05:15:28.534575 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:28.534620 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:28.535750 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:28.536144 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:28.536149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:28.536529 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:28.536781 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.536786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.537453 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.537453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.538292 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:28.538294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.538640 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.538641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:28.538745 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.539399 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:28.539408 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:28.540832 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:28.542249 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.542256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.542258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.542487 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.542680 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:28.542681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:28.542681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:28.542784 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:28.543049 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:28.543053 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:28.543183 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 05:15:28.543272 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 05:15:28.543402 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260925 05:15:28.544183 19980 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38171
I20260925 05:15:28.544305 19980 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:28.544492 19980 heartbeater.cc:511] Master 127.13.160.190:38171 requested a full tablet report, sending...
I20260925 05:15:28.544888 19521 ts_manager.cc:194] Registered new tserver with Master: 5cab56c0a2604bcf9fd6a6916c4c3eb6 (127.13.160.131:38097)
I20260925 05:15:28.545252 19521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:57683
I20260925 05:15:28.547324 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/test-admin.keytab.
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.563998 13954 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 05:15:28.565789 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:28.564975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.13.160.129:37217)
0925 05:15:28.565159 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.565163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.565179 (+ 16us) server_negotiation.cc:407] Connection header received
0925 05:15:28.565207 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.565209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.565222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.565299 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.565683 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39022: BlockingRecv error: recv got EOF from 127.0.0.1:39022 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
W20260925 05:15:28.565913 13954 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc' of type FILE
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.567118 13954 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.568041 13954 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:28.575752 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.568580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.13.160.129:37217)
0925 05:15:28.568636 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.568638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.568684 (+ 46us) server_negotiation.cc:407] Connection header received
0925 05:15:28.568732 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.568733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.568744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.568791 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.569171 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.570009 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.570713 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.571118 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.571180 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.573242 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.573251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.573251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.573259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.574674 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.575058 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.575061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.575062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.575115 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.575260 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.575260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.575260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.575391 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.575491 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.575552 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.575615 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 05:15:28.577610 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:28.577065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.13.160.129:37217)
0925 05:15:28.577124 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.577126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.577178 (+ 52us) server_negotiation.cc:407] Connection header received
0925 05:15:28.577229 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.577230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.577238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.577285 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.577542 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39052: BlockingRecv error: recv got EOF from 127.0.0.1:39052 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 05:15:28.577702 13954 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc' of type FILE
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.578718 13954 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.579587 13954 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:28.586884 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.580104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.13.160.129:37217)
0925 05:15:28.580162 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.580164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.580222 (+ 58us) server_negotiation.cc:407] Connection header received
0925 05:15:28.580295 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.580296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.580307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.580374 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.580742 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.581233 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.581923 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.582135 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.582245 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.584479 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.584481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.584481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.584490 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.585696 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.586100 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.586101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.586102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.586162 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.586368 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.586369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.586369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.586480 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.586588 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.586688 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.586771 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 05:15:28.588522 19987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:28.587910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.13.160.129:37217)
0925 05:15:28.587967 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.587969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.588039 (+ 70us) server_negotiation.cc:407] Connection header received
0925 05:15:28.588109 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.588110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.588120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.588172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.588445 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39080: BlockingRecv error: recv got EOF from 127.0.0.1:39080 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 05:15:28.588696 13954 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790313300911901-13954-0/krb5kdc/krb5cc)
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.589790 13954 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:28.590652 13954 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:15:28.597507 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:28.591233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.13.160.129:37217)
0925 05:15:28.591269 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:28.591271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:28.591359 (+ 88us) server_negotiation.cc:407] Connection header received
0925 05:15:28.591394 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:28.591394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:28.591403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:28.591446 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:28.591818 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.592294 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.593007 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:28.593186 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:28.593235 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:28.595262 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:28.595263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:28.595263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:28.595270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:28.596649 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.596949 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.596950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.596950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.596997 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:28.597147 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:28.597147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:28.597148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:28.597234 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:28.597299 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:28.597386 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 05:15:28.597442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:15:28.598317 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19583
I20260925 05:15:28.603106 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19717
I20260925 05:15:28.607376 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19851
I20260925 05:15:28.611810 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 19489
2026-09-25T05:15:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1869 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[20000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[20000](info): set up 2 sockets
Sep 25 05:15:28 dist-test-slave-9tgz krb5kdc[20000](info): commencing operation
krb5kdc: starting...
W20260925 05:15:29.668004 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:29 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab.
Entry for principal oryx/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab.
I20260925 05:15:29.787703 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:44123
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab
--principal=oryx/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:44123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:29.871378 20013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:29.871616 20013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:29.871641 20013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:29.872956 20013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:29.873034 20013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:29.873067 20013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:29.873085 20013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:29.873106 20013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:29.874722 20013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:44123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab
--principal=oryx/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:44123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:29.875073 20013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:29.875324 20013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:29.880281 20018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:29.880262 20019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:29.880262 20021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:29.880475 20013 server_base.cc:1034] running on GCE node
I20260925 05:15:29.880728 20013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:29.881070 20013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:29.882249 20013 hybrid_clock.cc:648] HybridClock initialized: now 1790313329882247 us; error 44 us; skew 500 ppm
Sep 25 05:15:29 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:29.888340 20013 init.cc:377] Logged in from keytab as oryx/127.13.160.190@KRBTEST.COM (short username oryx)
I20260925 05:15:29.888921 20013 webserver.cc:460] Webserver started at http://127.13.160.190:36857/ using document root <none> and password file <none>
I20260925 05:15:29.889247 20013 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:29.889333 20013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:29.889482 20013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:29.890616 20013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "a2b329fde8734929a8cf5d69c3fdc2e3"
format_stamp: "Formatted at 2026-09-25 05:15:29 on dist-test-slave-9tgz"
server_key: "ab296b26702290c29326688f2eed5545"
server_key_iv: "511e15b0fad8e2f68b4b01f26cc24ae3"
server_key_version: "encryptionkey@0"
I20260925 05:15:29.891044 20013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "a2b329fde8734929a8cf5d69c3fdc2e3"
format_stamp: "Formatted at 2026-09-25 05:15:29 on dist-test-slave-9tgz"
server_key: "ab296b26702290c29326688f2eed5545"
server_key_iv: "511e15b0fad8e2f68b4b01f26cc24ae3"
server_key_version: "encryptionkey@0"
I20260925 05:15:29.892505 20013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:29.893263 20028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:29.893452 20013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:29.893563 20013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "a2b329fde8734929a8cf5d69c3fdc2e3"
format_stamp: "Formatted at 2026-09-25 05:15:29 on dist-test-slave-9tgz"
server_key: "ab296b26702290c29326688f2eed5545"
server_key_iv: "511e15b0fad8e2f68b4b01f26cc24ae3"
server_key_version: "encryptionkey@0"
I20260925 05:15:29.893654 20013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:29.907683 20013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:29.909777 20013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:29.909957 20013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:29.914204 20013 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:44123
I20260925 05:15:29.914240 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:44123 every 8 connection(s)
I20260925 05:15:29.914611 20013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:29.915668 20081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:29.918120 20081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Bootstrap starting.
I20260925 05:15:29.918762 20081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:29.919088 20081 log.cc:824] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:29.919852 20081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: No bootstrap required, opened a new log
I20260925 05:15:29.921489 20081 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } }
I20260925 05:15:29.921653 20081 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:29.921679 20081 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b329fde8734929a8cf5d69c3fdc2e3, State: Initialized, Role: FOLLOWER
I20260925 05:15:29.921869 20081 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } }
I20260925 05:15:29.921936 20081 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:29.921958 20081 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:29.922031 20081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:29.923033 20081 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } }
I20260925 05:15:29.923058 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20013
I20260925 05:15:29.923173 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:29.923166 20081 leader_election.cc:302] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2b329fde8734929a8cf5d69c3fdc2e3; no voters:
I20260925 05:15:29.923352 20081 leader_election.cc:288] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:29.923385 13954 external_mini_cluster.cc:1428] Setting key 8103410c5a08bae8b90c42a504c77f6f
I20260925 05:15:29.923421 20084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:29.923585 20084 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b329fde8734929a8cf5d69c3fdc2e3, State: Running, Role: LEADER
I20260925 05:15:29.923650 20081 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:29.923683 20084 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } }
I20260925 05:15:29.924006 20084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b329fde8734929a8cf5d69c3fdc2e3. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } }
I20260925 05:15:29.924098 20084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:29.924021 20085 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } }
I20260925 05:15:29.924255 20085 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:29.924605 20092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:29.925424 20092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:29.927181 20092 catalog_manager.cc:1347] Generated new cluster ID: dcf272ed2e844283b2bb5dd57c20aa84
I20260925 05:15:29.927238 20092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:29 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:29.936247 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:29.924043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.13.160.190:44123)
0925 05:15:29.924824 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:29.924836 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:29.924873 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:15:29.925019 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:29.925027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:29.925109 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:29.925299 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:29.926068 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:29.927448 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:29.928511 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:29.929177 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:29.929299 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:29.932204 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:29.932209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:29.932209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:29.932223 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:29.934435 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:29.934862 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:29.934865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:29.934866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:29.935021 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:29.935247 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:29.935247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:29.935248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:29.935585 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:29.935743 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:29.935858 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 05:15:29.935998 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":70,"threads_started":1}
I20260925 05:15:29.943107 20092 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:29.943732 20092 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:29.951624 20092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Generated new TSK 0
I20260925 05:15:29.951856 20092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
I20260925 05:15:29.989856 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:30.098088 20107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:30.098337 20107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:30.098364 20107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:30.099671 20107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:30.099725 20107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:30.099781 20107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:30.101357 20107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:30.101640 20107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:30.101903 20107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.107326 20112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.107326 20113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.107560 20107 server_base.cc:1034] running on GCE node
W20260925 05:15:30.107318 20115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.107831 20107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.108109 20107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.109264 20107 hybrid_clock.cc:648] HybridClock initialized: now 1790313330109258 us; error 33 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.112751 20107 init.cc:377] Logged in from keytab as oryx/127.13.160.129@KRBTEST.COM (short username oryx)
I20260925 05:15:30.113242 20107 webserver.cc:460] Webserver started at http://127.13.160.129:41235/ using document root <none> and password file <none>
I20260925 05:15:30.113495 20107 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.113569 20107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.113710 20107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:30.115149 20107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "95a917e3c1a54f388d980e986957a79e"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "a4de8aa9291e1e7d39889fcb9985467d"
server_key_iv: "85cefccec9606222086b2deacbcd419b"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.115547 20107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "95a917e3c1a54f388d980e986957a79e"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "a4de8aa9291e1e7d39889fcb9985467d"
server_key_iv: "85cefccec9606222086b2deacbcd419b"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.117504 20107 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:30.118474 20122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.118667 20107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:30.118775 20107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "95a917e3c1a54f388d980e986957a79e"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "a4de8aa9291e1e7d39889fcb9985467d"
server_key_iv: "85cefccec9606222086b2deacbcd419b"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.118871 20107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.138211 20107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.140303 20107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:30.140486 20107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.140750 20107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:30.141090 20107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:30.141145 20107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.141211 20107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:30.141242 20107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.147492 20107 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:44139
I20260925 05:15:30.147557 20235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:44139 every 8 connection(s)
I20260925 05:15:30.147928 20107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:30.156119 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20107
I20260925 05:15:30.156242 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:30.156471 13954 external_mini_cluster.cc:1428] Setting key 8ef4a0830334345713a2b5e1b3af6c57
I20260925 05:15:30.158208 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.148729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:41885 (local address 127.13.160.190:44123)
0925 05:15:30.148786 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.148790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.148995 (+ 205us) server_negotiation.cc:407] Connection header received
0925 05:15:30.149478 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.149479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.149492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.149563 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.150548 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.151142 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.152232 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.152443 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.152515 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.154773 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.154774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.154775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.154783 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.156546 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.157063 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.157064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.157064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.157145 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.157487 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.157488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.157488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.157614 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.157675 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.158043 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.158138 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:30.158528 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.148680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:41885)
0925 05:15:30.148841 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.148896 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.149392 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.149645 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.149649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.150245 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.150455 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.150459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.151262 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.151263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.152119 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.152121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.152535 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.152536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.152651 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.153199 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.153208 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.154669 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.156656 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.156666 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.156669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.156987 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.157245 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.157246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.157247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.157414 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.157686 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.157693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.157894 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.158019 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.158224 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260925 05:15:30.159309 20236 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:30.159436 20236 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:30.159667 20236 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:30.160305 20045 ts_manager.cc:194] Registered new tserver with Master: 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
I20260925 05:15:30.161262 20045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:41885
WARNING: no policy specified for oryx/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
I20260925 05:15:30.185886 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:30.270299 20241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:30.270605 20241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:30.270661 20241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:30.271914 20241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:30.271996 20241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:30.272042 20241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:30.273607 20241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:30.273933 20241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:30.274199 20241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.279642 20247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.279642 20249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.279713 20246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.279930 20241 server_base.cc:1034] running on GCE node
I20260925 05:15:30.280151 20241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.280483 20241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.281673 20241 hybrid_clock.cc:648] HybridClock initialized: now 1790313330281659 us; error 30 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.284324 20241 init.cc:377] Logged in from keytab as oryx/127.13.160.130@KRBTEST.COM (short username oryx)
I20260925 05:15:30.284799 20241 webserver.cc:460] Webserver started at http://127.13.160.130:45291/ using document root <none> and password file <none>
I20260925 05:15:30.285048 20241 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.285120 20241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.285279 20241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:30.286377 20241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "73299027d2b8646330b6c851f47f1b4d"
server_key_iv: "86bb3a88a92c822daf68a5864b775465"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.286757 20241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "73299027d2b8646330b6c851f47f1b4d"
server_key_iv: "86bb3a88a92c822daf68a5864b775465"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.288143 20241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:30.288909 20256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.289131 20241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:15:30.289230 20241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "73299027d2b8646330b6c851f47f1b4d"
server_key_iv: "86bb3a88a92c822daf68a5864b775465"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.289321 20241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.304335 20241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.306329 20241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:30.306510 20241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.306787 20241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:30.307163 20241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:30.307220 20241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.307261 20241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:30.307304 20241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.313167 20241 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:39381
I20260925 05:15:30.313237 20369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:39381 every 8 connection(s)
I20260925 05:15:30.313627 20241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:30.321261 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20241
I20260925 05:15:30.321345 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:30.321537 13954 external_mini_cluster.cc:1428] Setting key 5903ba0df8924e491a9ce27bde553167
I20260925 05:15:30.323184 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.314424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:53559 (local address 127.13.160.190:44123)
0925 05:15:30.314512 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.314515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.314714 (+ 199us) server_negotiation.cc:407] Connection header received
0925 05:15:30.315383 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.315384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.315394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.315448 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.316424 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.316947 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.317909 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.318093 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.318145 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.320263 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.320265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.320265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.320273 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.321562 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.322027 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.322028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.322029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.322094 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.322430 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.322431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.322431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.322577 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.322635 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.323006 (+ 371us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.323107 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 05:15:30.323514 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.314388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:53559)
0925 05:15:30.314590 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.314627 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.315291 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.315510 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.315513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.316132 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.316333 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.316338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.317038 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.317039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.317833 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.317835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.318166 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.318167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.318272 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.318750 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.318758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.320173 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.321649 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.321657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.321660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.321948 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.322211 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.322212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.322212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.322357 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.322649 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.322655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.322847 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.322988 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.323222 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260925 05:15:30.324060 20370 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:30.324180 20370 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:30.324398 20370 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:30.324887 20045 ts_manager.cc:194] Registered new tserver with Master: 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130:39381)
I20260925 05:15:30.325296 20045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:53559
WARNING: no policy specified for oryx/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
I20260925 05:15:30.349148 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:30.432662 20375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:30.432961 20375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:30.433002 20375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:30.434311 20375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:30.434382 20375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:30.434428 20375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:30.436040 20375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:44123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:30.436344 20375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:30.436589 20375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.441684 20381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.441679 20380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.441694 20383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.441745 20375 server_base.cc:1034] running on GCE node
I20260925 05:15:30.442107 20375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.442320 20375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.443486 20375 hybrid_clock.cc:648] HybridClock initialized: now 1790313330443466 us; error 36 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.446061 20375 init.cc:377] Logged in from keytab as oryx/127.13.160.131@KRBTEST.COM (short username oryx)
I20260925 05:15:30.446528 20375 webserver.cc:460] Webserver started at http://127.13.160.131:39721/ using document root <none> and password file <none>
I20260925 05:15:30.446789 20375 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.446861 20375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.447064 20375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:30.448137 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "57b9d139d2df4f3584173c1c04608085"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "1ddae35c162d598826573d3311458e9d"
server_key_iv: "82337c0dc97198731f33c3769c5c0b01"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.448518 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "57b9d139d2df4f3584173c1c04608085"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "1ddae35c162d598826573d3311458e9d"
server_key_iv: "82337c0dc97198731f33c3769c5c0b01"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.449853 20375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:30.450580 20390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.450758 20375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:30.450860 20375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "57b9d139d2df4f3584173c1c04608085"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "1ddae35c162d598826573d3311458e9d"
server_key_iv: "82337c0dc97198731f33c3769c5c0b01"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.450978 20375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.473730 20375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.476044 20375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:30.476225 20375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.476477 20375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:30.476820 20375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:30.476876 20375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.476922 20375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:30.476944 20375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.482795 20375 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:32777
I20260925 05:15:30.482836 20503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:32777 every 8 connection(s)
I20260925 05:15:30.483206 20375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:30.484236 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20375
I20260925 05:15:30.484332 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:30.484515 13954 external_mini_cluster.cc:1428] Setting key 37f0c9763c0773a20c7d17193b6fa4b7
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:30.492880 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.484021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:44961 (local address 127.13.160.190:44123)
0925 05:15:30.484105 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.484108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.484296 (+ 188us) server_negotiation.cc:407] Connection header received
0925 05:15:30.484921 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.484922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.484935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.484994 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.486109 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.486618 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.487679 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.487868 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.487925 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.490079 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.490080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.490081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.490089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.491593 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.491978 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.491978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.491979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.492079 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.492315 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.492315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.492316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.492430 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.492481 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.492748 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.492813 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:15:30.493165 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.484002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:44961)
0925 05:15:30.484152 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.484196 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.484798 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.485075 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.485080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.485808 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.486014 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.486019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.486702 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.486702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.487592 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.487595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.487945 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.487946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.488050 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.488548 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.488556 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.489980 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.491673 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.491680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.491682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.491910 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.492147 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.492148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.492148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.492256 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.492484 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.492488 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.492616 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.492717 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.492846 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260925 05:15:30.493731 20504 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:30.493844 20504 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:30.494050 20504 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:30.494472 20045 ts_manager.cc:194] Registered new tserver with Master: 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:30.494827 20045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:44961
I20260925 05:15:30.496151 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.160.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.189.keytab.
Entry for principal oryx/127.13.160.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.189.keytab.
I20260925 05:15:30.522383 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:46643
--webserver_interface=127.13.160.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.189.keytab
--principal=oryx/127.13.160.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:44123,127.13.160.189:46643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:30.605183 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:30.605429 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:30.605453 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:30.606727 20509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:30.606779 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:30.606794 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:30.606807 20509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:30.606859 20509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:30.608523 20509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:44123,127.13.160.189:46643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.189.keytab
--principal=oryx/127.13.160.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:46643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.160.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
W20260925 05:15:30.608675 20509 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.160.190:44123,127.13.160.189:46643'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 05:15:30.608834 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:30.609081 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.614033 20517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.614056 20514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.614089 20515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.614293 20509 server_base.cc:1034] running on GCE node
I20260925 05:15:30.614550 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.614853 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.616065 20509 hybrid_clock.cc:648] HybridClock initialized: now 1790313330616053 us; error 45 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.618593 20509 init.cc:377] Logged in from keytab as oryx/127.13.160.189@KRBTEST.COM (short username oryx)
I20260925 05:15:30.619238 20509 webserver.cc:460] Webserver started at http://127.13.160.189:38751/ using document root <none> and password file <none>
I20260925 05:15:30.619495 20509 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.619565 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.619705 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:30.620826 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/instance:
uuid: "46b559b325994b509ce1b821e583156a"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "c451b661a2059a939738c27c195784f6"
server_key_iv: "433c2dddefac8771b18583f4436ad22c"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.621198 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal/instance:
uuid: "46b559b325994b509ce1b821e583156a"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "c451b661a2059a939738c27c195784f6"
server_key_iv: "433c2dddefac8771b18583f4436ad22c"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.622730 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:30.623674 20524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.623878 20509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:30.623986 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "46b559b325994b509ce1b821e583156a"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "c451b661a2059a939738c27c195784f6"
server_key_iv: "433c2dddefac8771b18583f4436ad22c"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.624073 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.634557 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.636550 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:30.636720 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.640875 20509 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.189:46643
I20260925 05:15:30.640906 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.189:46643 every 8 connection(s)
I20260925 05:15:30.641275 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
I20260925 05:15:30.642266 20577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:30.643918 20577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:15:30.647724 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20509
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.189@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
I20260925 05:15:30.653694 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.644720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.13.160.190:44123)
0925 05:15:30.644783 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.644785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.645032 (+ 247us) server_negotiation.cc:407] Connection header received
0925 05:15:30.645494 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.645495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.645506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.645577 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.646183 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.646712 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.647868 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.648090 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.648179 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.650502 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.650504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.650504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.650514 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.652190 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.652612 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.652614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.652614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.652676 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.652931 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.652932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.652932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.653089 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.653160 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.653504 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.653599 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 05:15:30.653879 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.644708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52948)
0925 05:15:30.644881 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.644925 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.645404 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.645634 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.645638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.645890 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.646096 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.646101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.646829 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.646830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.647744 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.647747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.648193 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.648194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.648383 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.648986 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.648995 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.650410 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.652287 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.652294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.652296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.652525 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.652744 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.652744 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.652745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.652855 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.653162 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.653168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.653350 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.653470 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.653636 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260925 05:15:30.654701 20577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:15:30.656875 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.648264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.13.160.189:46643)
0925 05:15:30.648436 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.648441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.648507 (+ 66us) server_negotiation.cc:407] Connection header received
0925 05:15:30.648584 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.648587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.648611 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.648687 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.649288 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.649951 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.650919 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.651421 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.651486 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.653788 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.653797 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.653797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.653806 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.655644 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.656017 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.656020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.656020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.656090 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.656301 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.656302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.656302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.656447 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.656631 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.656675 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.656751 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.189@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
I20260925 05:15:30.661986 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.655023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54174)
0925 05:15:30.655075 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.655085 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.655127 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.655364 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.655365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.655570 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.655612 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.655613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.656291 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.656291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.656787 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.656788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.657245 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.657246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.657307 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.657690 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.657694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.658801 (+ 1107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.660965 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.660966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.660967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.661140 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.661345 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.661346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.661346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.661399 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.661655 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.661656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.661693 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.661762 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.661865 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 05:15:30.662030 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.655096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.13.160.189:46643)
0925 05:15:30.655263 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.655264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.655270 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:15:30.655283 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.655284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.655291 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.655363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.655693 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.656204 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.656878 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.657172 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.657241 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.658916 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.658918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.658918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.658925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.660862 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.661222 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.661224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.661224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.661281 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.661467 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.661468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.661468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.661590 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.661666 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.661787 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.661914 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260925 05:15:30.663532 20577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Bootstrap starting.
I20260925 05:15:30.664136 20577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:30.664356 20577 log.cc:824] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:30.665043 20577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: No bootstrap required, opened a new log
I20260925 05:15:30.666405 20577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } }
I20260925 05:15:30.666530 20577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:30.666553 20577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b559b325994b509ce1b821e583156a, State: Initialized, Role: FOLLOWER
I20260925 05:15:30.666685 20577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } }
I20260925 05:15:30.667029 20577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:30.666986 20584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } }
I20260925 05:15:30.667104 20584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:15:30.668728 20595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:15:30.668793 20595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:15:30.670364 20509 master_runner.cc:428] Detected that this master 46b559b325994b509ce1b821e583156a is joining an existing cluster
I20260925 05:15:30.670404 20509 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.160.189:46643
I20260925 05:15:30.679008 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.671451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54178)
0925 05:15:30.671701 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.671709 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.671776 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.671906 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.671907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.672152 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.672221 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.672223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.673139 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.673140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.674016 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.674018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.674424 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.674425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.674486 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.675015 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.675019 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.675420 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.677453 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.677454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.677455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.677954 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.678178 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.678180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.678267 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.678550 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.678551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.678592 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.678677 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.678787 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":107,"threads_started":1}
I20260925 05:15:30.679008 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.671643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.13.160.189:46643)
0925 05:15:30.671680 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.671682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.671755 (+ 73us) server_negotiation.cc:407] Connection header received
0925 05:15:30.671829 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.671829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.671837 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.671885 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.672311 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.673046 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.674100 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.674312 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.674403 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.675509 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.675510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.675511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.675518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.677370 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678036 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.678037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.678037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.678093 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678343 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.678344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.678345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.678478 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.678559 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.678698 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.678836 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":19}
I20260925 05:15:30.679899 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.671471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.13.160.190:44123)
0925 05:15:30.671518 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.671520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.671673 (+ 153us) server_negotiation.cc:407] Connection header received
0925 05:15:30.671772 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.671773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.671784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.671836 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.672261 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.673048 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.675179 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.675480 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.675550 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.676737 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.676739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.676739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.676748 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.678351 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678714 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.678715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.678715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.678773 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.679034 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.679035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.679035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.679166 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.679211 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.679723 (+ 512us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.679809 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:30.680297 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.671437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52958)
0925 05:15:30.671608 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.671619 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.671704 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.671884 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.671885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.672115 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.672179 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.672181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.674119 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.674120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.675091 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.675093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.675550 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.675551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.675626 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.676282 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.676287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.676659 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.678418 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.678419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.678625 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.678843 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.678844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.678844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.678957 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.679562 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.679563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.679605 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.680090 (+ 485us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.680190 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.190@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
I20260925 05:15:30.689668 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.682373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54184)
0925 05:15:30.682520 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.682562 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.682658 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.682796 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.682800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.683156 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.683206 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.683208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.683863 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.683864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.684469 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.684471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.684841 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.684842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.684944 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.685423 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.685427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.686993 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.688623 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.688630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.688633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.688832 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.689026 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.689026 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.689026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.689083 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.689352 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.689358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.689405 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.689472 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.689568 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260925 05:15:30.689697 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.682462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.13.160.189:46643)
0925 05:15:30.682504 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.682506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.682632 (+ 126us) server_negotiation.cc:407] Connection header received
0925 05:15:30.682724 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.682725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.682735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.682791 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.683291 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.683776 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.684569 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.684760 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.684851 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.687071 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.687073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.687073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.687080 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.688540 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.688902 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.688903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.688903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.688961 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.689141 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.689142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.689142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.689276 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.689357 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.689516 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.689601 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:30.690382 20045 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.160.189:46643: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "46b559b325994b509ce1b821e583156a"
member_type: NON_VOTER
last_known_addr {
host: "127.13.160.189"
port: 46643
}
attrs {
promote: true
}
}
dest_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3"
cas_config_opid_index: -1
I20260925 05:15:30.690743 20045 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } }
I20260925 05:15:30.691287 20605 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 46b559b325994b509ce1b821e583156a (127.13.160.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:30.691620 20551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:30.692392 20605 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.160.189:46643
I20260925 05:15:30.692368 20551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 05:15:30.692649 20509 master.cc:537] Master@127.13.160.189:46643 shutting down...
I20260925 05:15:30.692735 20605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 05:15:30.693385 20030 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260925 05:15:30.693789 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:30.693866 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
W20260925 05:15:30.693876 20030 consensus_peers.cc:487] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 -> Peer 46b559b325994b509ce1b821e583156a (127.13.160.189:46643): Couldn't send request to peer 46b559b325994b509ce1b821e583156a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 05:15:30.695740 20509 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:30.695828 20509 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:30.695894 20509 tablet_replica.cc:330] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: stopping tablet replica
I20260925 05:15:30.708734 20509 master.cc:552] Master@127.13.160.189:46643 shutdown complete.
I20260925 05:15:30.708789 20509 master_runner.cc:305] Clearing existing system tablet
I20260925 05:15:30.708935 20509 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.708983 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.709649 20509 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:30.710171 20610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.710335 20509 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.710395 20509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.710427 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "46b559b325994b509ce1b821e583156a"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "c451b661a2059a939738c27c195784f6"
server_key_iv: "433c2dddefac8771b18583f4436ad22c"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.711655 20509 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:30.713258 20509 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 05:15:30.713321 20509 log.cc:1196] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 05:15:30.713542 20509 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Deleting consensus metadata
I20260925 05:15:30.713668 20509 master_runner.cc:315] Copying system tablet from 127.13.160.190:44123
I20260925 05:15:30.714300 20509 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: tablet copy: Beginning tablet copy session from remote peer at address 127.13.160.190:44123
I20260925 05:15:30.720556 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.714720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.13.160.190:44123)
0925 05:15:30.714774 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.714776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.714845 (+ 69us) server_negotiation.cc:407] Connection header received
0925 05:15:30.714942 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.714943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.714954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.715001 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.715329 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.715845 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.716585 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.716799 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.716905 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.717891 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.717892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.717893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.717902 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.719381 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.719754 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.719755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.719756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.719815 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.720008 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.720008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.720009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.720130 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.720202 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.720377 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.720462 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 05:15:30.720579 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.714657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52968)
0925 05:15:30.714776 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.714788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.714861 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.715015 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.715016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.715191 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.715244 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.715245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.715937 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.715938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.716490 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.716491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.716905 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.716906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.716973 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.717468 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.717474 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.717794 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.719483 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.719484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.719484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.719673 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.719879 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.719880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.719880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.719936 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.720205 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.720207 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.720251 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.720350 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.720460 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260925 05:15:30.721141 20065 tablet_copy_service.cc:140] P a2b329fde8734929a8cf5d69c3fdc2e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 46b559b325994b509ce1b821e583156a ({username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:52968)
I20260925 05:15:30.721261 20065 tablet_copy_service.cc:161] P a2b329fde8734929a8cf5d69c3fdc2e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 46b559b325994b509ce1b821e583156a at {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:52968: session id = 46b559b325994b509ce1b821e583156a-00000000000000000000000000000000
I20260925 05:15:30.722151 20065 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 05:15:30.722862 20509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 05:15:30.724270 20509 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: tablet copy: Starting download of 0 data blocks...
I20260925 05:15:30.724431 20509 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: tablet copy: Starting download of 1 WAL segments...
I20260925 05:15:30.725418 20509 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 05:15:30.726410 20065 tablet_copy_service.cc:342] P a2b329fde8734929a8cf5d69c3fdc2e3: Request end of tablet copy session 46b559b325994b509ce1b821e583156a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:52968
I20260925 05:15:30.726472 20065 tablet_copy_service.cc:433] P a2b329fde8734929a8cf5d69c3fdc2e3: ending tablet copy session 46b559b325994b509ce1b821e583156a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 46b559b325994b509ce1b821e583156a
I20260925 05:15:30.728924 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.730087 20618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.730168 20620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.730259 20509 server_base.cc:1034] running on GCE node
W20260925 05:15:30.730091 20617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.730556 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.730719 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.731854 20509 hybrid_clock.cc:648] HybridClock initialized: now 1790313330731846 us; error 25 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.733392 20509 init.cc:377] Logged in from keytab as oryx/127.13.160.189@KRBTEST.COM (short username oryx)
I20260925 05:15:30.733736 20509 webserver.cc:460] Webserver started at http://127.13.160.189:38229/ using document root <none> and password file <none>
I20260925 05:15:30.733861 20509 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.733922 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.734556 20509 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.735105 20627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.735268 20509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.735332 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "46b559b325994b509ce1b821e583156a"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "c451b661a2059a939738c27c195784f6"
server_key_iv: "433c2dddefac8771b18583f4436ad22c"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.735390 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.759287 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.761021 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.764235 20509 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.189:46643
I20260925 05:15:30.764572 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.189:46643 every 8 connection(s)
I20260925 05:15:30.764654 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
I20260925 05:15:30.765403 20680 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:15:30.765854 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Bootstrap starting.
I20260925 05:15:30.768175 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:15:30.768390 20680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Bootstrap complete.
I20260925 05:15:30.768607 20680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } }
I20260925 05:15:30.768677 20680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 46b559b325994b509ce1b821e583156a, State: Initialized, Role: LEARNER
I20260925 05:15:30.768703 20680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } }
I20260925 05:15:30.768909 20680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:30.768882 20683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:30.768953 20683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: LEARNER
I20260925 05:15:30.770399 20694 catalog_manager.cc:1259] Loaded cluster ID: dcf272ed2e844283b2bb5dd57c20aa84
I20260925 05:15:30.770701 20694 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: loading cluster ID for follower catalog manager: success
I20260925 05:15:30.771544 20694 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: acquiring CA information for follower catalog manager: success
I20260925 05:15:30.771826 20694 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:15:30.816126 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.809255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.13.160.189:46643)
0925 05:15:30.809418 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.809420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.809427 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:15:30.809450 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.809450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.809460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.809537 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.810141 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.810958 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.811761 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.811968 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.812071 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.813368 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.813370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.813370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.813378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.814882 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.815314 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.815315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.815316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.815377 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.815590 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.815591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.815591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.815708 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.815814 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.815914 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.816002 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for oryx/127.13.160.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.188.keytab.
Entry for principal oryx/127.13.160.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.188.keytab.
I20260925 05:15:30.841990 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:33765
--webserver_interface=127.13.160.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.188.keytab
--principal=oryx/127.13.160.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:30.926254 20698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:30.926491 20698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:30.926518 20698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:30.927753 20698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:30.927802 20698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:30.927817 20698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:30.927830 20698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:30.927842 20698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:30.929339 20698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.188.keytab
--principal=oryx/127.13.160.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:33765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.160.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:30.929557 20698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:30.929755 20698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:30.934602 20706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.934616 20704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:30.934598 20703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:30.934651 20698 server_base.cc:1034] running on GCE node
I20260925 05:15:30.935138 20698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:30.935467 20698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:30.936661 20698 hybrid_clock.cc:648] HybridClock initialized: now 1790313330936655 us; error 37 us; skew 500 ppm
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:30.939194 20698 init.cc:377] Logged in from keytab as oryx/127.13.160.188@KRBTEST.COM (short username oryx)
I20260925 05:15:30.939713 20698 webserver.cc:460] Webserver started at http://127.13.160.188:34663/ using document root <none> and password file <none>
I20260925 05:15:30.939962 20698 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:30.940033 20698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:30.940188 20698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:30.941316 20698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/instance:
uuid: "991b79e8819b451ba3665c30a72cd897"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "2c3d0737314650ac145a6e0cd2c966df"
server_key_iv: "681c102756b52dd1a73454597531b510"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.941687 20698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal/instance:
uuid: "991b79e8819b451ba3665c30a72cd897"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "2c3d0737314650ac145a6e0cd2c966df"
server_key_iv: "681c102756b52dd1a73454597531b510"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.943104 20698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:30.943864 20713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:30.944049 20698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:30.944159 20698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "991b79e8819b451ba3665c30a72cd897"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "2c3d0737314650ac145a6e0cd2c966df"
server_key_iv: "681c102756b52dd1a73454597531b510"
server_key_version: "encryptionkey@0"
I20260925 05:15:30.944233 20698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:30.960903 20698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:30.962960 20698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:30.963141 20698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:30.967191 20698 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.188:33765
I20260925 05:15:30.967243 20765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.188:33765 every 8 connection(s)
I20260925 05:15:30.967644 20698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
I20260925 05:15:30.968644 20766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:30.970474 20766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.188@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:30.977382 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20698
I20260925 05:15:30.979925 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.971384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.13.160.190:44123)
0925 05:15:30.971477 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.971479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.971582 (+ 103us) server_negotiation.cc:407] Connection header received
0925 05:15:30.972100 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.972101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.972112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.972170 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.972808 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.973298 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.974266 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.974444 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.974494 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.976832 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.976834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.976834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.976841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.978431 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.978853 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.978855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.978855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.978953 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.979224 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.979224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.979225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.979356 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.979449 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.979746 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.979832 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 05:15:30.980142 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.971265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52972)
0925 05:15:30.971461 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.971501 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.971960 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.972258 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.972262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.972526 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.972730 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.972734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.973372 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.973373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.974188 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.974190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.974530 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.974531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.974648 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.975296 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.975307 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.976738 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.978532 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.978539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.978541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.978771 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.979031 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.979032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.979032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.979152 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.979458 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.979464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.979606 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.979721 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.979885 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260925 05:15:30.980747 20766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.188@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
I20260925 05:15:30.986690 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.977960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.13.160.188:33765)
0925 05:15:30.978150 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.978154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.978172 (+ 18us) server_negotiation.cc:407] Connection header received
0925 05:15:30.978189 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.978191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.978202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.978283 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.978852 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.979590 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.980524 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.980932 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.981001 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.983710 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.983719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.983719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.983726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.985455 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.985813 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.985818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.985818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.985876 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.986080 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.986081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.986082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.986249 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.986407 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.986480 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.986558 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260925 05:15:30.988190 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.981034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54212)
0925 05:15:30.981078 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.981088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.981118 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.981299 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.981300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.981436 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.981477 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.981478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.982158 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.982159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.982877 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.982879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.983318 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.983319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.983416 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.983846 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.983851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.985372 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.987143 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.987143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.987144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.987356 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.987557 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.987558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.987558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.987613 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.987869 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.987871 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.987911 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.987995 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.988096 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 05:15:30.988190 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.981113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.13.160.189:46643)
0925 05:15:30.981179 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.981181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.981189 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:30.981212 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.981213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.981232 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.981296 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.981554 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.982034 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.982990 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.983245 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.983314 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.985462 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.985463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.985464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.985471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.987054 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.987419 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.987420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.987420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.987478 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.987684 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.987685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.987685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.987795 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.987866 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.988019 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.988098 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:15:30.988641 20766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:15:30 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313330, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.188@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
I20260925 05:15:30.995064 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.988944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.13.160.188:33765)
0925 05:15:30.988974 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:30.988976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:30.988984 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:30.989055 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:30.989055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:30.989065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:30.989119 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:30.989355 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.989839 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.990501 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.990714 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.990796 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.992608 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:30.992609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:30.992610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:30.992616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:30.994010 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.994300 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.994301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.994301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.994341 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:30.994485 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:30.994485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:30.994485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:30.994605 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:30.994652 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:30.994816 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 05:15:30.994988 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":16}
I20260925 05:15:30.995050 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:30.988891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.0.0.1:39238)
0925 05:15:30.988937 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:30.988948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:30.988991 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:30.989121 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:30.989122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:30.989253 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:30.989281 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.989282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.989931 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.989932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.990407 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:30.990409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:30.990794 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:30.990795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:30.990841 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:30.991341 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:30.991346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:30.992535 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:30.994091 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.994092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.994093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.994245 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.994392 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:30.994393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:30.994393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:30.994440 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:30.994684 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:30.994686 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:30.994726 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 05:15:30.994798 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 05:15:30.994954 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 05:15:30.996654 20766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Bootstrap starting.
I20260925 05:15:30.997520 20766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:30.997813 20766 log.cc:824] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:30.998606 20766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: No bootstrap required, opened a new log
I20260925 05:15:31.000128 20766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } }
I20260925 05:15:31.000279 20766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:31.000329 20766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991b79e8819b451ba3665c30a72cd897, State: Initialized, Role: FOLLOWER
I20260925 05:15:31.000486 20766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } }
I20260925 05:15:31.000816 20766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:31.000793 20771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } } }
I20260925 05:15:31.000931 20771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:15:31.002552 20782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:15:31.002625 20782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:15:31.004923 20698 master_runner.cc:428] Detected that this master 991b79e8819b451ba3665c30a72cd897 is joining an existing cluster
I20260925 05:15:31.004994 20698 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.160.188:33765
I20260925 05:15:31.013227 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.13.160.188:33765)
0925 05:15:31.006124 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.006127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.006462 (+ 335us) server_negotiation.cc:407] Connection header received
0925 05:15:31.006544 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.006545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.006554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.006581 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.007172 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.007894 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009067 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009340 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009377 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.010521 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.010529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.010529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.010536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.012084 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.012435 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.012437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.012437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.012493 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.012696 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.012697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.012697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.012812 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.012887 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.013008 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.013097 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:15:31.013310 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.0.0.1:39240)
0925 05:15:31.006406 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.006415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.006484 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.006631 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.006631 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.006981 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.007087 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.007089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.007980 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.007981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.008968 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.008970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009467 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.009525 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.010057 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.010062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.010413 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.012182 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.012183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.012184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.012354 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.012574 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.012574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.012575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.012627 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.012876 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.012877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.012916 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.012984 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.013148 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":103,"thread_start_us":104,"threads_started":1}
I20260925 05:15:31.038261 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20107
I20260925 05:15:31.044191 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.13.160.189:46643)
0925 05:15:31.006547 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.006549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.006675 (+ 126us) server_negotiation.cc:407] Connection header received
0925 05:15:31.006786 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.006787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.006797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.006827 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.007305 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.008060 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009117 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009415 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009459 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.041156 (+ 31697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.041158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.041159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.041167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.042854 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.043306 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.043308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.043308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.043363 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.043551 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.043552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.043552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.043701 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.043743 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.043985 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.044078 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:15:31.044615 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:44139
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:44123
--webserver_port=41235
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:15:31.045053 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.13.160.190:44123)
0925 05:15:31.006461 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.006463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.006471 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:31.006496 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.006497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.006509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.006566 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.007181 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.007916 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009070 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009375 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009412 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.041307 (+ 31895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.041308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.041309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.041317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.043678 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.044142 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.044144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.044144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.044202 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.044419 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.044421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.044421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.044563 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.044605 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.044816 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.044897 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 05:15:31.045447 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54222)
0925 05:15:31.006616 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.006626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.006711 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.006889 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.006890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.007141 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.007208 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.007210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.008158 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.008159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.009019 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.009021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009535 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.009611 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.040452 (+ 30841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.040459 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.041015 (+ 556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.042995 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.042996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.042997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.043221 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.043432 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.043432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.043433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.043488 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.043790 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.043791 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.043848 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.044922 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.045046 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":81,"threads_started":1}
I20260925 05:15:31.045460 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.006038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52984)
0925 05:15:31.006268 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.006279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.006332 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.006615 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.006616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.006984 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.007087 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.007089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.008000 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.008001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.008968 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.008970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.009481 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.009481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.009545 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.040643 (+ 31098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.040649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.041174 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.043786 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.043787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.043787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.044055 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.044284 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.044285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.044286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.044344 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.044656 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.044656 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.044703 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.045140 (+ 437us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.045279 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":45,"thread_start_us":79,"threads_started":1}
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313329, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.190@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
I20260925 05:15:31.056505 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.048121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.0.0.1:39248)
0925 05:15:31.048170 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.048180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.048213 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.048347 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.048348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.048517 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.048568 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.048569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.049552 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.049553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.050394 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.050397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.050945 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.050946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.051024 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.051531 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.051536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.053165 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.055296 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.055297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.055298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.055531 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.055768 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.055769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.055769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.055842 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.056165 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.056166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.056215 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.056283 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.056397 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 05:15:31.056504 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.048158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.13.160.188:33765)
0925 05:15:31.048195 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.048197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.048206 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:31.048275 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.048276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.048287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.048344 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.048659 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.049455 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.050490 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.050831 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.050938 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.053263 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.053265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.053266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.053274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.055200 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.055620 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.055622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.055622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.055689 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.055915 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.055916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.055916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.056088 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.056165 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.056329 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.056404 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:15:31.057466 20045 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.160.188:33765: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "991b79e8819b451ba3665c30a72cd897"
member_type: NON_VOTER
last_known_addr {
host: "127.13.160.188"
port: 33765
}
attrs {
promote: true
}
}
dest_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3"
cas_config_opid_index: 5
I20260925 05:15:31.057740 20045 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
I20260925 05:15:31.058143 20604 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 991b79e8819b451ba3665c30a72cd897 (127.13.160.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.059280 20740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:31.060220 20740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 05:15:31.060509 20605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:31.061261 20604 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.160.188:33765
I20260925 05:15:31.061342 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.061414 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.061698 20698 master.cc:537] Master@127.13.160.188:33765 shutting down...
I20260925 05:15:31.063014 20740 raft_consensus.cc:858] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
W20260925 05:15:31.063110 20740 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
I20260925 05:15:31.063684 20740 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 46b559b325994b509ce1b821e583156a (127.13.160.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:31.065045 20771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b329fde8734929a8cf5d69c3fdc2e3. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:31.065132 20771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260925 05:15:31.065167 20771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:31.065199 20771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260925 05:15:31.065223 20771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } }
I20260925 05:15:31.065275 20771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260925 05:15:31.065876 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.058235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54238)
0925 05:15:31.058523 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.058535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.058574 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.059232 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.059233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.059235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.059279 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.059282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.060330 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.060331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.062236 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.062239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.065576 (+ 3337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.065577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.065664 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.065695 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.065746 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.065797 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
I20260925 05:15:31.066198 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.058968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.13.160.189:46643)
0925 05:15:31.059032 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.059034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.059043 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:31.059069 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.059070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.059096 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.059162 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.059351 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.060122 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.062347 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.065260 (+ 2913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.065324 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.065412 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.066081 (+ 669us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.066131 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:31.066663 20654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 LEARNER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 05:15:31.066936 20605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 05:15:31.067238 20792 mvcc.cc:205] Tried to move back new op lower bound from 7333123399480225792 to 7333123399367856128. Current Snapshot: MvccSnapshot[applied={T|T < 7333123399378350080}]
I20260925 05:15:31.067585 20654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 991b79e8819b451ba3665c30a72cd897 (127.13.160.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.068720 20800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.068863 20800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: LEARNER
I20260925 05:15:31.069032 20683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b329fde8734929a8cf5d69c3fdc2e3. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: NON_VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: true } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.069123 20683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: LEARNER
I20260925 05:15:31.069662 20604 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: attempting to promote NON_VOTER 46b559b325994b509ce1b821e583156a to VOTER
I20260925 05:15:31.069859 20604 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
I20260925 05:15:31.070351 20654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 LEARNER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 05:15:31.070524 20605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260925 05:15:31.070626 20029 consensus_peers.cc:487] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 -> Peer 991b79e8819b451ba3665c30a72cd897 (127.13.160.188:33765): Couldn't send request to peer 991b79e8819b451ba3665c30a72cd897. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 05:15:31.071173 20605 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 46b559b325994b509ce1b821e583156a (127.13.160.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.072054 20605 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.072125 20605 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.072171 20605 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.072214 20605 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.072706 20654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 46b559b325994b509ce1b821e583156a (127.13.160.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.073091 20698 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260925 05:15:31.073199 20698 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260925 05:15:31.073236 20698 tablet_replica.cc:330] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: stopping tablet replica
I20260925 05:15:31.073676 20683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.073763 20683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:15:31.088519 20698 master.cc:552] Master@127.13.160.188:33765 shutdown complete.
I20260925 05:15:31.088603 20698 master_runner.cc:305] Clearing existing system tablet
I20260925 05:15:31.088745 20698 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:31.088801 20698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:31.089622 20698 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:15:31.090200 20802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.090309 20698 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:31.090360 20698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.090382 20698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "991b79e8819b451ba3665c30a72cd897"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "2c3d0737314650ac145a6e0cd2c966df"
server_key_iv: "681c102756b52dd1a73454597531b510"
server_key_version: "encryptionkey@0"
I20260925 05:15:31.091233 20698 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:31.092923 20698 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 05:15:31.092998 20698 log.cc:1196] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 05:15:31.093245 20698 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Deleting consensus metadata
I20260925 05:15:31.093362 20698 master_runner.cc:315] Copying system tablet from 127.13.160.190:44123
I20260925 05:15:31.093968 20698 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: tablet copy: Beginning tablet copy session from remote peer at address 127.13.160.190:44123
I20260925 05:15:31.101686 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.094293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:52994)
0925 05:15:31.094425 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.094436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.094506 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.094677 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.094678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.094965 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.095026 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.095028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.095966 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.095967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.096846 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.096849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.097383 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.097384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.097454 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.098059 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.098064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.098476 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.100429 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.100431 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.100431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.100684 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.100908 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.100909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.100909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.100978 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.101334 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.101335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.101393 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.101466 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.101579 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260925 05:15:31.102174 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.094364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.13.160.190:44123)
0925 05:15:31.094415 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.094418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.094506 (+ 88us) server_negotiation.cc:407] Connection header received
0925 05:15:31.094591 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.094592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.094603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.094659 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.095109 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.095875 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.096926 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.097231 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.097321 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.098581 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.098583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.098583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.098600 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.100335 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.100758 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.100760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.100760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.100828 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.101104 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.101105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.101105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.101261 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.101340 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.101601 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.101688 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:31.102632 20065 tablet_copy_service.cc:140] P a2b329fde8734929a8cf5d69c3fdc2e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 991b79e8819b451ba3665c30a72cd897 ({username='oryx', principal='oryx/127.13.160.188@KRBTEST.COM'} at 127.0.0.1:52994)
I20260925 05:15:31.102697 20065 tablet_copy_service.cc:161] P a2b329fde8734929a8cf5d69c3fdc2e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 991b79e8819b451ba3665c30a72cd897 at {username='oryx', principal='oryx/127.13.160.188@KRBTEST.COM'} at 127.0.0.1:52994: session id = 991b79e8819b451ba3665c30a72cd897-00000000000000000000000000000000
I20260925 05:15:31.103389 20065 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 05:15:31.103929 20698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 05:15:31.105288 20698 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: tablet copy: Starting download of 0 data blocks...
I20260925 05:15:31.105438 20698 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: tablet copy: Starting download of 1 WAL segments...
I20260925 05:15:31.106289 20698 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 05:15:31.107548 20065 tablet_copy_service.cc:342] P a2b329fde8734929a8cf5d69c3fdc2e3: Request end of tablet copy session 991b79e8819b451ba3665c30a72cd897-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.160.188@KRBTEST.COM'} at 127.0.0.1:52994
I20260925 05:15:31.107615 20065 tablet_copy_service.cc:433] P a2b329fde8734929a8cf5d69c3fdc2e3: ending tablet copy session 991b79e8819b451ba3665c30a72cd897-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 991b79e8819b451ba3665c30a72cd897
I20260925 05:15:31.110355 20698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:31.111591 20809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.111748 20812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:31.112319 20698 server_base.cc:1034] running on GCE node
W20260925 05:15:31.115361 20810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:31.115599 20698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:31.115763 20698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:31.116895 20698 hybrid_clock.cc:648] HybridClock initialized: now 1790313331116888 us; error 30 us; skew 500 ppm
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:31.118535 20698 init.cc:377] Logged in from keytab as oryx/127.13.160.188@KRBTEST.COM (short username oryx)
I20260925 05:15:31.118806 20698 webserver.cc:460] Webserver started at http://127.13.160.188:43099/ using document root <none> and password file <none>
I20260925 05:15:31.118968 20698 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:31.119009 20698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:31.119868 20698 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:31.120505 20819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.120695 20698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:31.120747 20698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "991b79e8819b451ba3665c30a72cd897"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "2c3d0737314650ac145a6e0cd2c966df"
server_key_iv: "681c102756b52dd1a73454597531b510"
server_key_version: "encryptionkey@0"
I20260925 05:15:31.120807 20698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:31.134712 20698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:31.137001 20698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:31.140151 20698 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.188:33765
I20260925 05:15:31.140280 20871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.188:33765 every 8 connection(s)
I20260925 05:15:31.140513 20698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
I20260925 05:15:31.141600 20872 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:15:31.142143 20872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Bootstrap starting.
I20260925 05:15:31.144781 20872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:15:31.145007 20872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: Bootstrap complete.
I20260925 05:15:31.145224 20872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
I20260925 05:15:31.145337 20872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991b79e8819b451ba3665c30a72cd897, State: Initialized, Role: LEARNER
I20260925 05:15:31.145375 20872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } }
I20260925 05:15:31.145557 20872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:31.147030 20886 catalog_manager.cc:1259] Loaded cluster ID: dcf272ed2e844283b2bb5dd57c20aa84
I20260925 05:15:31.147145 20886 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: loading cluster ID for follower catalog manager: success
I20260925 05:15:31.148140 20875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.148242 20875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: LEARNER
I20260925 05:15:31.148478 20886 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: acquiring CA information for follower catalog manager: success
I20260925 05:15:31.148815 20886 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 05:15:31.154424 20791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:31.154680 20791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:31.154722 20791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:31.156057 20791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:31.156131 20791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:31.156181 20791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:31.157841 20791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:44139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=41235
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:31.158139 20791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:31.158402 20791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:31.163697 20891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.163702 20894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.164459 20892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:31.164933 20791 server_base.cc:1034] running on GCE node
I20260925 05:15:31.165176 20791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:31.165454 20791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:31.166633 20791 hybrid_clock.cc:648] HybridClock initialized: now 1790313331166621 us; error 34 us; skew 500 ppm
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:31.169696 20791 init.cc:377] Logged in from keytab as oryx/127.13.160.129@KRBTEST.COM (short username oryx)
I20260925 05:15:31.170220 20791 webserver.cc:460] Webserver started at http://127.13.160.129:41235/ using document root <none> and password file <none>
I20260925 05:15:31.170518 20791 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:31.170588 20791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:31.172123 20791 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:31.172776 20901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.172957 20791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:31.173069 20791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "95a917e3c1a54f388d980e986957a79e"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "a4de8aa9291e1e7d39889fcb9985467d"
server_key_iv: "85cefccec9606222086b2deacbcd419b"
server_key_version: "encryptionkey@0"
I20260925 05:15:31.173460 20791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:31.195742 20791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:31.198176 20791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:31.198374 20791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:31.198668 20791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:31.199107 20791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:31.199168 20791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.199215 20791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:31.199268 20791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.205730 20791 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:44139
I20260925 05:15:31.205780 21014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:44139 every 8 connection(s)
I20260925 05:15:31.206184 20791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:31.211097 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 20791
I20260925 05:15:31.211262 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20241
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:31.218353 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:39381
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:44123
--webserver_port=45291
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
I20260925 05:15:31.219939 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:57223 (local address 127.13.160.189:46643)
0925 05:15:31.207061 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.207063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.207402 (+ 339us) server_negotiation.cc:407] Connection header received
0925 05:15:31.208628 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.208629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.208645 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.208689 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.210110 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.210786 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.212600 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.212811 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.212909 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.216199 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.216200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.216201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.216210 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.218075 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.218672 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.218674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.218674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.218757 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.219104 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.219106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.219106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.219277 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.219337 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.219745 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.219831 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:31.220265 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:48899 (local address 127.13.160.190:44123)
0925 05:15:31.207447 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.207449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.207459 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:31.208624 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.208625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.208639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.208678 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.210184 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.211263 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.213332 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.213676 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.213722 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.217218 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.217220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.217220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.217228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.218976 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.219381 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.219382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.219382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.219445 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.219650 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.219651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.219651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.219793 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.219834 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.220051 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.220150 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 05:15:31.220515 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.13.160.129:57223)
0925 05:15:31.207232 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.207278 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.208502 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.208839 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.208843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.209916 (+ 1073us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.210002 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.210004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.210964 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.210965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.212473 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.212476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.212920 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.212921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.213078 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.213685 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.213694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.216080 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.218175 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.218184 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.218187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.218565 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.218837 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.218838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.218838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.219012 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.219376 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.219383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.219562 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.220054 (+ 492us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.220248 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260925 05:15:31.220664 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:48899)
0925 05:15:31.207232 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.207277 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.208478 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.208838 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.208843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.209926 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.210045 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.210047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.212278 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.212279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.213233 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.213236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.213802 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.213803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.213878 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.214352 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.214358 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.217122 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.219081 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.219082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.219082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.219297 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.219513 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.219514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.219514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.219570 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.219878 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.219878 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.219919 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.220393 (+ 474us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.220489 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260925 05:15:31.221026 21016 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:31.221036 21017 heartbeater.cc:347] Connected to a master server at 127.13.160.189:46643
I20260925 05:15:31.221149 21016 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.221408 21016 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:31.221148 21017 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.221760 21017 heartbeater.cc:511] Master 127.13.160.189:46643 requested a full tablet report, sending...
I20260925 05:15:31.221884 20045 ts_manager.cc:194] Re-registered known tserver with Master: 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
I20260925 05:15:31.222195 20644 ts_manager.cc:194] Registered new tserver with Master: 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
I20260925 05:15:31.222388 20045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:48899
I20260925 05:15:31.224059 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:59145 (local address 127.13.160.188:33765)
0925 05:15:31.207147 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.207150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.207550 (+ 400us) server_negotiation.cc:407] Connection header received
0925 05:15:31.208766 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.208768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.208781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.208891 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.209911 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.215654 (+ 5743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.216833 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.217757 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.220410 (+ 2653us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.220525 (+ 115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.220526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.220526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.220532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.222541 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.223058 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.223059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.223059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.223129 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.223360 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.223361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.223361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.223502 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.223588 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.223853 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.223945 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
I20260925 05:15:31.224066 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.207368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.13.160.129:59145)
0925 05:15:31.207490 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.207500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.208489 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.208944 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.208944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.209477 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.209786 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.209793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.215810 (+ 6017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.215811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.216734 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.216737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.217875 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.217877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.217964 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.218655 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.218660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.220313 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.222651 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.222652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.222652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.222948 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.223216 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.223217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.223217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.223282 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.223591 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.223592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.223716 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.223811 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.223962 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260925 05:15:31.224459 21015 heartbeater.cc:347] Connected to a master server at 127.13.160.188:33765
I20260925 05:15:31.224521 21015 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.224578 21015 heartbeater.cc:511] Master 127.13.160.188:33765 requested a full tablet report, sending...
I20260925 05:15:31.224959 20828 ts_manager.cc:194] Registered new tserver with Master: 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
W20260925 05:15:31.308947 21023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:31.309231 21023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:31.309276 21023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:31.310536 21023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:31.310626 21023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:31.310675 21023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:31.312371 21023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:39381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=45291
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:31.312695 21023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:31.312963 21023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:31.318384 21029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.318414 21031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.318384 21028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:31.318629 21023 server_base.cc:1034] running on GCE node
I20260925 05:15:31.318843 21023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:31.319132 21023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:31.320343 21023 hybrid_clock.cc:648] HybridClock initialized: now 1790313331320331 us; error 26 us; skew 500 ppm
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:31.323182 21023 init.cc:377] Logged in from keytab as oryx/127.13.160.130@KRBTEST.COM (short username oryx)
I20260925 05:15:31.323597 21023 webserver.cc:460] Webserver started at http://127.13.160.130:45291/ using document root <none> and password file <none>
I20260925 05:15:31.323791 21023 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:31.323848 21023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:31.325340 21023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:31.325975 21038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.326176 21023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:15:31.326267 21023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "73299027d2b8646330b6c851f47f1b4d"
server_key_iv: "86bb3a88a92c822daf68a5864b775465"
server_key_version: "encryptionkey@0"
I20260925 05:15:31.326606 21023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:31.337003 21023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:31.339071 21023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:31.339191 21023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:31.339396 21023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:31.339725 21023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:31.339759 21023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.339818 21023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:31.339841 21023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.345772 21023 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:39381
I20260925 05:15:31.345834 21151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:39381 every 8 connection(s)
I20260925 05:15:31.346220 21023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
I20260925 05:15:31.355990 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21023
I20260925 05:15:31.356165 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20375
I20260925 05:15:31.358729 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:50653 (local address 127.13.160.190:44123)
0925 05:15:31.347252 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.347254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.347545 (+ 291us) server_negotiation.cc:407] Connection header received
0925 05:15:31.348113 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.348114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.348127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.348161 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.348948 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349718 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.350849 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.351110 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.351146 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.353463 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.353465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.353465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.353474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.356843 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357537 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.357538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.357538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.357602 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357879 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.357880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.357880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.358024 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.358069 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.358491 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.358590 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:31.358736 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:57279 (local address 127.13.160.189:46643)
0925 05:15:31.347183 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.347185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.347563 (+ 378us) server_negotiation.cc:407] Connection header received
0925 05:15:31.348096 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.348097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.348111 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.348169 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.348942 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349447 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.350501 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.350702 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.350788 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.354613 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.354616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.354616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.354625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.356811 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357507 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.357508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.357509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.357572 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357864 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.357865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.357866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.358019 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.358075 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.358486 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.358586 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:15:31.358956 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:57119 (local address 127.13.160.188:33765)
0925 05:15:31.347328 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.347330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.347544 (+ 214us) server_negotiation.cc:407] Connection header received
0925 05:15:31.348113 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.348114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.348127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.348168 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.348954 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349712 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.350877 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.351240 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.351341 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.354682 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.354684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.354684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.354693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.357732 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.358109 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.358110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.358111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.358182 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.358378 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.358379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.358379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.358507 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.358547 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.358737 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.358827 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 05:15:31.359439 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.13.160.130:57119)
0925 05:15:31.347389 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.347459 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.348016 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.348639 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.348640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.348793 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.348864 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.348865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349814 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.349815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.350746 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.350748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.351343 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.351344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.351414 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.351928 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.351932 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.354523 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.357815 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.357817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.358018 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.358254 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.358255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.358255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.358307 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.358594 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.358594 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.358630 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.359027 (+ 397us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.359236 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":151,"thread_start_us":100,"threads_started":1}
I20260925 05:15:31.359440 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:50653)
0925 05:15:31.347451 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.347460 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.348011 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.348262 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.348265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.348683 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.348833 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.348839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349814 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.349815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.350753 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.350754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.351209 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.351209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.351300 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.351750 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.351755 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.353306 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.357110 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.357111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.357435 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.357691 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.357692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.357799 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.358115 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.358121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.358317 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.358927 (+ 610us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.359178 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":159,"thread_start_us":113,"threads_started":1}
I20260925 05:15:31.359439 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.347096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.13.160.130:57279)
0925 05:15:31.347455 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.347464 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.347991 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.348260 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.348264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.348627 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.348831 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.348836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.349572 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.349574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.350395 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.350397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.350798 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.350799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.350995 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.351625 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.351634 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.354519 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.356919 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.356931 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.356934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.357415 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.357645 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.357646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.357646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.357795 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.358122 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.358123 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.358317 (+ 194us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.358829 (+ 512us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.359082 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":221,"thread_start_us":65,"threads_started":1}
I20260925 05:15:31.359988 21154 heartbeater.cc:347] Connected to a master server at 127.13.160.189:46643
I20260925 05:15:31.360095 21154 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.360296 21154 heartbeater.cc:511] Master 127.13.160.189:46643 requested a full tablet report, sending...
I20260925 05:15:31.360764 20644 ts_manager.cc:194] Registered new tserver with Master: 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130:39381)
I20260925 05:15:31.363118 21152 heartbeater.cc:347] Connected to a master server at 127.13.160.188:33765
I20260925 05:15:31.363173 21152 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.363262 21153 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:31.363266 21152 heartbeater.cc:511] Master 127.13.160.188:33765 requested a full tablet report, sending...
I20260925 05:15:31.363305 21153 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.363355 21153 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:31.363605 20045 ts_manager.cc:194] Re-registered known tserver with Master: 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130:39381)
I20260925 05:15:31.363626 20828 ts_manager.cc:194] Registered new tserver with Master: 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130:39381)
I20260925 05:15:31.364126 20045 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:50653
I20260925 05:15:31.364288 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:32777
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:44123
--webserver_port=39721
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:31.452648 21159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:31.452883 21159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:31.452909 21159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:31.454074 21159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:31.454113 21159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:31.454139 21159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:31.455672 21159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:32777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=39721
--tserver_master_addrs=127.13.160.190:44123,127.13.160.189:46643,127.13.160.188:33765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:31.455900 21159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:31.456100 21159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:31.461333 21167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.461305 21165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:31.461311 21164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:31.461639 21159 server_base.cc:1034] running on GCE node
I20260925 05:15:31.461882 21159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:31.462169 21159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:31.463327 21159 hybrid_clock.cc:648] HybridClock initialized: now 1790313331463316 us; error 30 us; skew 500 ppm
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:31.465903 21159 init.cc:377] Logged in from keytab as oryx/127.13.160.131@KRBTEST.COM (short username oryx)
I20260925 05:15:31.466420 21159 webserver.cc:460] Webserver started at http://127.13.160.131:39721/ using document root <none> and password file <none>
I20260925 05:15:31.466662 21159 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:31.466748 21159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:31.468255 21159 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:31.468919 21174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.469125 21159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:31.469228 21159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "57b9d139d2df4f3584173c1c04608085"
format_stamp: "Formatted at 2026-09-25 05:15:30 on dist-test-slave-9tgz"
server_key: "1ddae35c162d598826573d3311458e9d"
server_key_iv: "82337c0dc97198731f33c3769c5c0b01"
server_key_version: "encryptionkey@0"
I20260925 05:15:31.469697 21159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:31.488289 21159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:31.490935 21159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:31.491127 21159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:31.491398 21159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:31.491739 21159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:31.491791 21159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.491827 21159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:31.491875 21159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:31.498132 21159 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:32777
I20260925 05:15:31.498178 21287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:32777 every 8 connection(s)
I20260925 05:15:31.498574 21159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:31.499673 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21159
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.189@KRBTEST.COM
Sep 25 05:15:31 dist-test-slave-9tgz krb5kdc[20000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313331, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.188@KRBTEST.COM
I20260925 05:15:31.512225 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.501015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.13.160.189:46643)
0925 05:15:31.501467 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.501469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.501707 (+ 238us) server_negotiation.cc:407] Connection header received
0925 05:15:31.501732 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.501733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.501745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.501798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.502573 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.503365 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506155 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.506461 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506501 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.508458 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.508459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.508460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.508468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.510396 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.510828 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.510830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.510830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.510914 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.511146 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.511146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.511147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.511295 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.511383 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.511990 (+ 607us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.512078 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":49,"threads_started":1}
I20260925 05:15:31.513240 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.500687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.13.160.190:44123)
0925 05:15:31.501530 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.501533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.501540 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:15:31.501559 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.501560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.501573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.502298 (+ 725us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.502584 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.503370 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504468 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.505767 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506078 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.508634 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.508635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.508635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.508643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.511710 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.512133 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.512134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.512134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.512197 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.512407 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.512407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.512408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.512550 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.513004 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.513056 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.513130 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":814,"thread_start_us":80,"threads_started":1}
I20260925 05:15:31.514765 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:51511 (local address 127.13.160.188:33765)
0925 05:15:31.499645 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.499648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.499919 (+ 271us) server_negotiation.cc:407] Connection header received
0925 05:15:31.500838 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.500839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.500853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.500886 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.505315 (+ 4429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.506023 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.508575 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.508894 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.508938 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.511166 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.511167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.511168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.511175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.513205 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513614 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.513616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.513616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.513672 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513950 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.513951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.513952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.514089 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.514140 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.514569 (+ 429us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.514652 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260925 05:15:31.514976 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:57179 (local address 127.13.160.190:44123)
0925 05:15:31.499756 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.499758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.500128 (+ 370us) server_negotiation.cc:407] Connection header received
0925 05:15:31.500991 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.500993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.501004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.501071 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.501992 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.502795 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504420 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.504767 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504826 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.510050 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.510051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.510051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.510059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.512847 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513492 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.513493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.513493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.513560 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.514216 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.514217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.514217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.514349 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.514390 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.514789 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.514871 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":193}
I20260925 05:15:31.515380 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.13.160.131:51511)
0925 05:15:31.499755 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.499805 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.500537 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.500964 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.500965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.503203 (+ 2238us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.504756 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.504757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.506249 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.508481 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.508484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.509019 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.509019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.509106 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.509526 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.509531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.511073 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.513302 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.513303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.513515 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.513740 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.513741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.513870 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.514175 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.514182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.514385 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.514859 (+ 474us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.515075 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":96,"threads_started":1}
I20260925 05:15:31.515379 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:57179)
0925 05:15:31.500040 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.500049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.500591 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.501193 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.501194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.501576 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.501883 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.501889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.502934 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.502935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.504321 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.504323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.504847 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.505000 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.506123 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.506128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.509888 (+ 3760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.512933 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.512943 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.512946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.513280 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.513634 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.513634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.513635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.513871 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.514444 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.514445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.514663 (+ 218us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.515123 (+ 460us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.515230 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":283,"thread_start_us":63,"threads_started":1}
I20260925 05:15:31.515847 21289 heartbeater.cc:347] Connected to a master server at 127.13.160.190:44123
I20260925 05:15:31.515960 21289 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.516153 21289 heartbeater.cc:511] Master 127.13.160.190:44123 requested a full tablet report, sending...
I20260925 05:15:31.515872 21288 heartbeater.cc:347] Connected to a master server at 127.13.160.188:33765
I20260925 05:15:31.516350 21288 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.516408 21288 heartbeater.cc:511] Master 127.13.160.188:33765 requested a full tablet report, sending...
I20260925 05:15:31.516562 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.501165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.13.160.188:33765)
0925 05:15:31.503646 (+ 2481us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.503648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.503656 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:31.503679 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.503680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.503692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.504430 (+ 738us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.504470 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.505204 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506176 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.506471 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.506790 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.509273 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.509274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.509274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.509281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.513699 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.514091 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.514092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.514092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.514140 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.514353 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.514354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.514354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.514693 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.516247 (+ 1554us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.516317 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.516438 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2458,"thread_start_us":55,"threads_started":1}
I20260925 05:15:31.516707 20044 ts_manager.cc:194] Re-registered known tserver with Master: 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:31.516742 20828 ts_manager.cc:194] Registered new tserver with Master: 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:31.517164 20044 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:57179
I20260925 05:15:31.515092 20045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:31.517467 20045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:31.521196 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:45449 (local address 127.13.160.189:46643)
0925 05:15:31.499709 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.499711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.499943 (+ 232us) server_negotiation.cc:407] Connection header received
0925 05:15:31.500666 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.500667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.500680 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.500712 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.501994 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.502795 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504426 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.504720 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504767 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.517769 (+ 13002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.517771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.517771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.517780 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.519606 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.520065 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.520066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.520067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.520130 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.520537 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.520538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.520538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.520684 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.520756 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.521015 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.521100 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 05:15:31.521220 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.499396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.13.160.131:45449)
0925 05:15:31.499810 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.499821 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.500541 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.500860 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.500864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.501576 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:31.501883 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.501889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.502936 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.502937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.504321 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.504323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.504847 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.504848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.505000 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.505827 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:31.505838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:31.517655 (+ 11817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:31.519674 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.519675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.519676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.519964 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.520208 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:31.520209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:31.520209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:31.520268 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:31.520809 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:31.520809 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:31.520871 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.520986 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.521118 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":45,"thread_start_us":86,"threads_started":1}
I20260925 05:15:31.522094 21290 heartbeater.cc:347] Connected to a master server at 127.13.160.189:46643
I20260925 05:15:31.522154 21290 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:31.522236 21290 heartbeater.cc:511] Master 127.13.160.189:46643 requested a full tablet report, sending...
I20260925 05:15:31.522541 20644 ts_manager.cc:194] Registered new tserver with Master: 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:31.526628 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.522435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47058 (local address 127.13.160.130:39381)
0925 05:15:31.522661 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.522666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.522677 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:31.522726 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.522728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.522745 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.522830 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.523116 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524285 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.525528 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526236 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.526327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.526386 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.526478 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.526531 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260925 05:15:31.526615 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.521805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:32777 (local address 127.0.0.1:46224)
0925 05:15:31.521842 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.521853 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.521890 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.522828 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.522829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.522831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.522880 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.522882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524394 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.524396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.525696 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.525699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526415 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.526415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.526475 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.526477 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.526504 (+ 27us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.526537 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":82}
I20260925 05:15:31.526810 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.522072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44139 (local address 127.0.0.1:48580)
0925 05:15:31.522178 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.522187 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.522222 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.523135 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.523136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.523138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.523176 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.523177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524482 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.524483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.525717 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.525719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526586 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.526587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.526651 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.526657 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.526688 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.526738 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260925 05:15:31.527031 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.522196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.13.160.131:32777)
0925 05:15:31.522338 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.522343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.522354 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:31.522727 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.522729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.522746 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.522823 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.523140 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524272 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.525799 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526323 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.526784 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.526856 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.526881 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.526947 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260925 05:15:31.526618 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.521895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39381 (local address 127.0.0.1:47058)
0925 05:15:31.522049 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.522059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.522101 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.522977 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.522978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.522980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.523031 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.523033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524666 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.524667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.525431 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.525433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526332 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.526334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.526404 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.526410 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.526464 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.526514 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260925 05:15:31.527674 21222 tablet_service.cc:1467] Processing CreateTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 (DEFAULT_TABLE table=test-table [id=632937e835d7490eb72ab230846ac0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:31.528041 21222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f158e6074a40b2ad299100c3ac9cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:31.531217 21314 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Bootstrap starting.
I20260925 05:15:31.531682 21086 tablet_service.cc:1467] Processing CreateTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 (DEFAULT_TABLE table=test-table [id=632937e835d7490eb72ab230846ac0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:31.531932 21314 tablet_bootstrap.cc:654] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:31.532105 21086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f158e6074a40b2ad299100c3ac9cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:31.532286 21314 log.cc:824] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:31.533066 21314 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: No bootstrap required, opened a new log
I20260925 05:15:31.533121 21314 ts_tablet_manager.cc:1397] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:31.535096 21316 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Bootstrap starting.
I20260925 05:15:31.535341 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.522179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.13.160.129:44139)
0925 05:15:31.522972 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.522976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.522985 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:31.523021 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.523022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.523051 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.523138 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.523231 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.524370 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.525799 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.526481 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.527047 (+ 566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.531145 (+ 4098us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.531179 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.535241 (+ 4062us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":98,"threads_started":1}
I20260925 05:15:31.535475 21314 raft_consensus.cc:348] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.535652 21314 raft_consensus.cc:374] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:31.535696 21314 raft_consensus.cc:729] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b9d139d2df4f3584173c1c04608085, State: Initialized, Role: FOLLOWER
I20260925 05:15:31.535777 21316 tablet_bootstrap.cc:654] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:31.535857 21314 consensus_queue.cc:260] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.536113 21316 log.cc:824] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:31.536170 21314 ts_tablet_manager.cc:1428] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:31.536284 21289 heartbeater.cc:503] Master 127.13.160.190:44123 was elected leader, sending a full tablet report...
I20260925 05:15:31.536131 20949 tablet_service.cc:1467] Processing CreateTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 (DEFAULT_TABLE table=test-table [id=632937e835d7490eb72ab230846ac0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:31.536562 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f158e6074a40b2ad299100c3ac9cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:31.537048 21316 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: No bootstrap required, opened a new log
I20260925 05:15:31.537127 21316 ts_tablet_manager.cc:1397] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:31.539194 21316 raft_consensus.cc:348] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.539397 21316 raft_consensus.cc:374] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:31.539448 21316 raft_consensus.cc:729] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, State: Initialized, Role: FOLLOWER
I20260925 05:15:31.539582 21316 consensus_queue.cc:260] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.539790 21319 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Bootstrap starting.
I20260925 05:15:31.539865 21316 ts_tablet_manager.cc:1428] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:31.539906 21153 heartbeater.cc:503] Master 127.13.160.190:44123 was elected leader, sending a full tablet report...
I20260925 05:15:31.540494 21319 tablet_bootstrap.cc:654] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:31.540819 21319 log.cc:824] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:31.541750 21319 tablet_bootstrap.cc:492] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: No bootstrap required, opened a new log
I20260925 05:15:31.541807 21319 ts_tablet_manager.cc:1397] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:31.543365 21319 raft_consensus.cc:348] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.543498 21319 raft_consensus.cc:374] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:31.543520 21319 raft_consensus.cc:729] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a917e3c1a54f388d980e986957a79e, State: Initialized, Role: FOLLOWER
I20260925 05:15:31.543670 21319 consensus_queue.cc:260] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.543926 21319 ts_tablet_manager.cc:1428] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:15:31.543998 21016 heartbeater.cc:503] Master 127.13.160.190:44123 was elected leader, sending a full tablet report...
W20260925 05:15:31.597121 21155 tablet.cc:2369] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:31.597621 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.594658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.0.0.1:39258)
0925 05:15:31.594747 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.594763 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.594835 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.595041 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.595042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.595044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.595105 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.595106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.595780 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.595782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.596619 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.596623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.597377 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.597378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.597442 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.597446 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.597488 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.597555 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 05:15:31.597610 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.594749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.13.160.188:33765)
0925 05:15:31.594825 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.594828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.594841 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:15:31.594922 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.594923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.594954 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.595023 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.595178 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.595678 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.596707 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.597276 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.597367 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.597471 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.597493 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.597532 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 05:15:31.597977 20846 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260925 05:15:31.598402 20875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b329fde8734929a8cf5d69c3fdc2e3. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.598522 20875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: LEARNER
I20260925 05:15:31.707121 21320 raft_consensus.cc:482] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:31.707283 21320 raft_consensus.cc:504] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.707677 21320 leader_election.cc:288] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777), 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
W20260925 05:15:31.708132 21018 tablet.cc:2369] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:31.710806 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.708144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44139 (local address 127.13.160.130:36701)
0925 05:15:31.708213 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.708222 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.708254 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.708448 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.708449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.708451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.708558 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.708560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.709213 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.709213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.709963 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.709965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.710542 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.710620 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.710624 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.710673 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.710703 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":4}
I20260925 05:15:31.710829 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.708261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:36701 (local address 127.13.160.129:44139)
0925 05:15:31.708307 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.708309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.708317 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:31.708345 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.708345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.708355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.708413 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.708639 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.709122 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710071 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.710447 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710519 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.710527 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.710705 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.710753 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:31.710907 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.708090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:53377 (local address 127.13.160.131:32777)
0925 05:15:31.708138 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.708140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.708186 (+ 46us) server_negotiation.cc:407] Connection header received
0925 05:15:31.708216 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.708217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.708226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.708276 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:31.708510 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.709040 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710107 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.710474 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710567 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.710575 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.710776 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.710818 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:31.710979 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.707996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:32777 (local address 127.13.160.130:53377)
0925 05:15:31.708065 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:31.708078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:31.708131 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:31.708330 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:31.708331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:31.708332 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:31.708423 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.708425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.709136 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.709137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.709963 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:31.709966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.710591 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.710592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:31.710689 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.710693 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:31.710758 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 05:15:31.710797 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":3}
I20260925 05:15:31.711374 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f158e6074a40b2ad299100c3ac9cb5" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b9d139d2df4f3584173c1c04608085" is_pre_election: true
I20260925 05:15:31.711373 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f158e6074a40b2ad299100c3ac9cb5" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a917e3c1a54f388d980e986957a79e" is_pre_election: true
I20260925 05:15:31.711549 21242 raft_consensus.cc:2454] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 0.
I20260925 05:15:31.711558 20969 raft_consensus.cc:2454] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 0.
I20260925 05:15:31.711800 21039 leader_election.cc:302] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, 95a917e3c1a54f388d980e986957a79e; no voters:
I20260925 05:15:31.711946 21320 raft_consensus.cc:2781] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:31.712000 21320 raft_consensus.cc:482] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:31.712023 21320 raft_consensus.cc:3037] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:31.712769 21320 raft_consensus.cc:504] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.712910 21320 leader_election.cc:288] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 election: Requested vote from peers 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777), 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139)
I20260925 05:15:31.713115 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f158e6074a40b2ad299100c3ac9cb5" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b9d139d2df4f3584173c1c04608085"
I20260925 05:15:31.713172 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f158e6074a40b2ad299100c3ac9cb5" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a917e3c1a54f388d980e986957a79e"
I20260925 05:15:31.713191 21242 raft_consensus.cc:3037] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:31.713267 20969 raft_consensus.cc:3037] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:31.713917 21242 raft_consensus.cc:2454] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 1.
I20260925 05:15:31.714097 21039 leader_election.cc:302] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57b9d139d2df4f3584173c1c04608085, 5c19be2fdd5f4eb19c0e2af73d4c6ea2; no voters:
I20260925 05:15:31.714211 21320 raft_consensus.cc:2781] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:31.714383 21320 raft_consensus.cc:686] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, State: Running, Role: LEADER
I20260925 05:15:31.714494 21320 consensus_queue.cc:237] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } }
I20260925 05:15:31.714620 20969 raft_consensus.cc:2454] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 1.
I20260925 05:15:31.715395 20045 catalog_manager.cc:5495] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:31.716001 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: NON_VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: true } } }
I20260925 05:15:31.716113 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.716974 21328 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: attempting to promote NON_VOTER 991b79e8819b451ba3665c30a72cd897 to VOTER
I20260925 05:15:31.717104 21328 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:31.717622 20654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 05:15:31.717674 20846 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 LEARNER]: Refusing update from remote peer a2b329fde8734929a8cf5d69c3fdc2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 05:15:31.717849 20606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 05:15:31.717967 20604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 05:15:31.718639 21328 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 991b79e8819b451ba3665c30a72cd897 (127.13.160.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.718812 20654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 991b79e8819b451ba3665c30a72cd897 (127.13.160.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.718770 20846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 991b79e8819b451ba3665c30a72cd897 (127.13.160.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719730 21328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719766 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719836 21328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.719877 20875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719928 21328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719960 20875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:15:31.719787 21330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:31.719846 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:31.720042 21330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:15:31.719998 21328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3 [sys.catalog]: This master's current role is: LEADER
W20260925 05:15:31.749429 21291 tablet.cc:2369] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:31.754343 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.752038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47070 (local address 127.13.160.130:39381)
0925 05:15:31.752109 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.752111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.752122 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:31.752163 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.752163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.752183 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.752251 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:31.752381 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.752890 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.753660 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.753844 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.753903 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.754050 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:31.754154 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:31.754223 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.754257 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.754279 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:15:31.756445 20969 raft_consensus.cc:1261] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Refusing update from remote peer 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:31.756445 21242 raft_consensus.cc:1261] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Refusing update from remote peer 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:31.756927 21320 consensus_queue.cc:1035] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:31.757113 21326 consensus_queue.cc:1035] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:31.759644 21334 mvcc.cc:205] Tried to move back new op lower bound from 7333123406871113728 to 7333123406703009792. Current Snapshot: MvccSnapshot[applied={T|T < 7333123406871113728}]
I20260925 05:15:31.763875 20045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:31.763983 20045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:31.765702 20045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2b329fde8734929a8cf5d69c3fdc2e3: Sending DeleteTablet for 3 replicas of tablet a1f158e6074a40b2ad299100c3ac9cb5
I20260925 05:15:31.766139 21222 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:31 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:46224
I20260925 05:15:31.766198 20949 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:31 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:48580
I20260925 05:15:31.766435 21086 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f158e6074a40b2ad299100c3ac9cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:31 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:47058
I20260925 05:15:31.766575 21343 tablet_replica.cc:330] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: stopping tablet replica
I20260925 05:15:31.766692 21343 raft_consensus.cc:2229] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:31.767068 21343 raft_consensus.cc:2258] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:31.767321 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20013
I20260925 05:15:31.767359 21342 tablet_replica.cc:330] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: stopping tablet replica
I20260925 05:15:31.768446 21343 ts_tablet_manager.cc:1905] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:31.770522 21343 ts_tablet_manager.cc:1918] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:31.770596 21343 log.cc:1196] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/a1f158e6074a40b2ad299100c3ac9cb5
I20260925 05:15:31.770848 21343 ts_tablet_manager.cc:1939] T a1f158e6074a40b2ad299100c3ac9cb5 P 95a917e3c1a54f388d980e986957a79e: Deleting consensus metadata
I20260925 05:15:31.771291 21342 raft_consensus.cc:2229] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:31.771426 21342 raft_consensus.cc:2258] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:31.771986 21344 tablet_replica.cc:330] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: stopping tablet replica
I20260925 05:15:31.772105 21344 raft_consensus.cc:2229] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:31.772244 21344 raft_consensus.cc:2258] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:31.772778 21344 ts_tablet_manager.cc:1905] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 05:15:31.774883 20904 connection.cc:535] server connection from 127.0.0.1:48580 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 05:15:31.774938 21344 ts_tablet_manager.cc:1918] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:31.775003 21344 log.cc:1196] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/a1f158e6074a40b2ad299100c3ac9cb5
I20260925 05:15:31.775271 21344 ts_tablet_manager.cc:1939] T a1f158e6074a40b2ad299100c3ac9cb5 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Deleting consensus metadata
W20260925 05:15:31.775696 21041 connection.cc:441] server connection from 127.0.0.1:47058 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47058 (request call id 1) could send its response
I20260925 05:15:31.777427 21342 ts_tablet_manager.cc:1905] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:31.779028 21342 ts_tablet_manager.cc:1918] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:31.779121 21342 log.cc:1196] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/a1f158e6074a40b2ad299100c3ac9cb5
I20260925 05:15:31.779377 21342 ts_tablet_manager.cc:1939] T a1f158e6074a40b2ad299100c3ac9cb5 P 57b9d139d2df4f3584173c1c04608085: Deleting consensus metadata
W20260925 05:15:31.779757 21175 connection.cc:441] server connection from 127.0.0.1:46224 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46224 (request call id 1) could send its response
I20260925 05:15:31.780712 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.767104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.13.160.188:33765)
0925 05:15:31.767145 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.767147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.771856 (+ 4709us) server_negotiation.cc:407] Connection header received
0925 05:15:31.771899 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.771900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.771913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.771987 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.772943 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.773714 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.774813 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.775153 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.775196 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.777263 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.777264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.777265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.777273 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.779398 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.779857 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.779858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.779859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.779915 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.780111 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.780112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.780112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.780240 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.780311 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.780522 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.780616 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:15:31.787465 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.777898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.13.160.188:33765)
0925 05:15:31.777968 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.777970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.779851 (+ 1881us) server_negotiation.cc:407] Connection header received
0925 05:15:31.779945 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.779947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.779958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.780006 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.780709 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.781481 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.782690 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.783031 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.783101 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.784390 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.784391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.784391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.784398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.786203 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.786612 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.786613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.786613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.786668 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.786853 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.786854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.786855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.787017 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.787124 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.787276 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.787361 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 05:15:31.787856 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:31.777856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.13.160.189:46643)
0925 05:15:31.777939 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:31.777941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:31.779977 (+ 2036us) server_negotiation.cc:407] Connection header received
0925 05:15:31.780086 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:31.780087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:31.780098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:31.780130 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:31.781097 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.781795 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.782790 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:31.783131 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:31.783196 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:31.784857 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:31.784859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:31.784859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:31.784868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:31.786497 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.786967 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.786969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.786969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.787028 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:31.787212 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:31.787213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:31.787213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:31.787337 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:31.787426 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:31.787667 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 05:15:31.787746 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 05:15:32.761353 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:32.761106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:59273)
0925 05:15:32.761229 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:32.761305 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
W20260925 05:15:32.761675 21016 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:44123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:32.762109 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:32.761624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:34589)
0925 05:15:32.761985 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:32.762059 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":66,"threads_started":1}
I20260925 05:15:32.762334 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:32.761873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:47925)
0925 05:15:32.762084 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:32.762285 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":49,"threads_started":1}
W20260925 05:15:32.762416 21153 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:44123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
W20260925 05:15:32.762506 21289 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:44123 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.266265 21356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b329fde8734929a8cf5d69c3fdc2e3)
I20260925 05:15:33.266427 21357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b329fde8734929a8cf5d69c3fdc2e3)
I20260925 05:15:33.266448 21356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:33.266557 21357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:33.266783 21356 leader_election.cc:288] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 46b559b325994b509ce1b821e583156a (127.13.160.189:46643)
I20260925 05:15:33.267017 21357 leader_election.cc:288] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 991b79e8819b451ba3665c30a72cd897 (127.13.160.188:33765)
I20260925 05:15:33.267252 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.266881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53494)
0925 05:15:33.267149 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.267201 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":74,"threads_started":1}
I20260925 05:15:33.267354 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.267044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53510)
0925 05:15:33.267247 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.267307 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":50,"threads_started":1}
W20260925 05:15:33.267377 20822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
W20260925 05:15:33.267633 20630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.267917 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.267695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53522)
0925 05:15:33.267818 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.267848 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
W20260925 05:15:33.268101 20822 leader_election.cc:334] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.268149 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.268039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53528)
0925 05:15:33.268093 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.268124 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 05:15:33.268268 20630 leader_election.cc:334] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.270507 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.267093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46643 (local address 127.0.0.1:54258)
0925 05:15:33.267274 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.267283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:33.267335 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:33.267534 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:33.267535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:33.267538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:33.267596 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:33.267598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.268570 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.268571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:33.269631 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:33.269633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.270252 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.270253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:33.270311 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:33.270314 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:33.270364 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 05:15:33.270430 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":36,"threads_started":1}
I20260925 05:15:33.270529 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.266951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:33765 (local address 127.0.0.1:39282)
0925 05:15:33.267131 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.267143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:33.267197 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:33.267607 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:33.267608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:33.267610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:33.267668 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:33.267670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.268828 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.268828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:33.269751 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:33.269752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.270298 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.270299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:33.270375 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:33.270379 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:33.270432 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 05:15:33.270466 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":36,"threads_started":1}
I20260925 05:15:33.270516 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.267167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.13.160.189:46643)
0925 05:15:33.267278 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:33.267280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:33.267384 (+ 104us) server_negotiation.cc:407] Connection header received
0925 05:15:33.267408 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:33.267409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:33.267420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:33.267482 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:33.267678 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.268475 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.269716 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.270173 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.270233 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:33.270239 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:33.270376 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 05:15:33.270443 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":38,"threads_started":1}
I20260925 05:15:33.270782 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.267062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.13.160.188:33765)
0925 05:15:33.267458 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:33.267460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:33.267468 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:33.267492 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:33.267493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:33.267503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:33.267753 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:33.267787 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.268749 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.269828 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:33.270195 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:33.270282 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:33.270291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:33.270691 (+ 400us) server_negotiation.cc:299] Negotiation successful
0925 05:15:33.270740 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":50,"threads_started":1}
I20260925 05:15:33.270964 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "991b79e8819b451ba3665c30a72cd897" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "46b559b325994b509ce1b821e583156a" is_pre_election: true
I20260925 05:15:33.270978 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46b559b325994b509ce1b821e583156a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "991b79e8819b451ba3665c30a72cd897" is_pre_election: true
I20260925 05:15:33.271076 20846 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46b559b325994b509ce1b821e583156a in term 1.
I20260925 05:15:33.271080 20654 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991b79e8819b451ba3665c30a72cd897 in term 1.
I20260925 05:15:33.271247 20628 leader_election.cc:302] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46b559b325994b509ce1b821e583156a, 991b79e8819b451ba3665c30a72cd897; no voters: a2b329fde8734929a8cf5d69c3fdc2e3
I20260925 05:15:33.271369 21357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 05:15:33.271420 21357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2b329fde8734929a8cf5d69c3fdc2e3)
I20260925 05:15:33.271406 20821 leader_election.cc:302] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46b559b325994b509ce1b821e583156a, 991b79e8819b451ba3665c30a72cd897; no voters: a2b329fde8734929a8cf5d69c3fdc2e3
I20260925 05:15:33.271464 21357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:15:33.271541 21356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 05:15:33.271591 21356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2b329fde8734929a8cf5d69c3fdc2e3)
I20260925 05:15:33.271623 21356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:15:33.272229 21357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:33.272274 21356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:33.272413 21357 leader_election.cc:288] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 election: Requested vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 991b79e8819b451ba3665c30a72cd897 (127.13.160.188:33765)
I20260925 05:15:33.272421 21356 leader_election.cc:288] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 election: Requested vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 46b559b325994b509ce1b821e583156a (127.13.160.189:46643)
I20260925 05:15:33.272573 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.272493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53532)
0925 05:15:33.272526 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.272552 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 05:15:33.272665 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.272553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53534)
0925 05:15:33.272609 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.272637 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 05:15:33.272724 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46b559b325994b509ce1b821e583156a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "991b79e8819b451ba3665c30a72cd897"
I20260925 05:15:33.272835 20846 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 46b559b325994b509ce1b821e583156a in current term 2: Already voted for candidate 991b79e8819b451ba3665c30a72cd897 in this term.
I20260925 05:15:33.273006 20654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "991b79e8819b451ba3665c30a72cd897" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "46b559b325994b509ce1b821e583156a"
I20260925 05:15:33.273072 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.272941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53554)
0925 05:15:33.272973 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.272998 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 05:15:33.273125 20654 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 991b79e8819b451ba3665c30a72cd897 in current term 2: Already voted for candidate 46b559b325994b509ce1b821e583156a in this term.
W20260925 05:15:33.273140 20630 leader_election.cc:334] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.273205 20630 leader_election.cc:302] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b559b325994b509ce1b821e583156a; no voters: 991b79e8819b451ba3665c30a72cd897, a2b329fde8734929a8cf5d69c3fdc2e3
I20260925 05:15:33.273265 21357 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 05:15:33.273223 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.272895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:53538)
0925 05:15:33.273167 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.273195 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255}
W20260925 05:15:33.273419 20822 leader_election.cc:334] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:33.273465 20822 leader_election.cc:302] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991b79e8819b451ba3665c30a72cd897; no voters: 46b559b325994b509ce1b821e583156a, a2b329fde8734929a8cf5d69c3fdc2e3
I20260925 05:15:33.273525 21356 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 05:15:33.762267 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.762000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:53457)
0925 05:15:33.762167 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.762215 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260925 05:15:33.763044 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.762807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:45541)
0925 05:15:33.762962 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.762999 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":59,"threads_started":1}
I20260925 05:15:33.763075 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:33.762727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:59187)
0925 05:15:33.763000 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:33.763032 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":67,"threads_started":1}
I20260925 05:15:34.763144 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.762831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:42057)
0925 05:15:34.763006 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.763081 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260925 05:15:34.763691 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.763453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:56101)
0925 05:15:34.763607 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.763641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":48,"threads_started":1}
I20260925 05:15:34.763707 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.763499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:33819)
0925 05:15:34.763598 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.763640 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":52,"threads_started":1}
I20260925 05:15:34.945562 21373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:34.945664 21373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:34.945892 21373 leader_election.cc:288] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 991b79e8819b451ba3665c30a72cd897 (127.13.160.188:33765)
I20260925 05:15:34.946269 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.946046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42632)
0925 05:15:34.946197 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.946232 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":48,"threads_started":1}
I20260925 05:15:34.946220 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46b559b325994b509ce1b821e583156a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "991b79e8819b451ba3665c30a72cd897" is_pre_election: true
I20260925 05:15:34.946352 20846 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46b559b325994b509ce1b821e583156a in term 2.
I20260925 05:15:34.946545 20628 leader_election.cc:302] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b559b325994b509ce1b821e583156a, 991b79e8819b451ba3665c30a72cd897; no voters:
I20260925 05:15:34.946637 21373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 05:15:34.946677 21373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:34.946710 21373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 2 FOLLOWER]: Advancing to term 3
I20260925 05:15:34.946877 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.946725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42658)
0925 05:15:34.946801 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.946848 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260925 05:15:34.946991 20630 leader_election.cc:334] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:34.947666 21373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:34.947808 21373 leader_election.cc:288] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 election: Requested vote from peers a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123), 991b79e8819b451ba3665c30a72cd897 (127.13.160.188:33765)
I20260925 05:15:34.948011 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.947892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42670)
0925 05:15:34.947947 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.947980 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 05:15:34.947983 20846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46b559b325994b509ce1b821e583156a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "991b79e8819b451ba3665c30a72cd897"
I20260925 05:15:34.948065 20846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 2 FOLLOWER]: Advancing to term 3
I20260925 05:15:34.948347 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:34.948249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42678)
0925 05:15:34.948299 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:34.948328 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 05:15:34.948434 20630 leader_election.cc:334] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
I20260925 05:15:34.948877 20846 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46b559b325994b509ce1b821e583156a in term 3.
I20260925 05:15:34.949085 20628 leader_election.cc:302] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46b559b325994b509ce1b821e583156a, 991b79e8819b451ba3665c30a72cd897; no voters: a2b329fde8734929a8cf5d69c3fdc2e3
I20260925 05:15:34.949210 21373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 3 FOLLOWER]: Leader election won for term 3
I20260925 05:15:34.949321 21373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [term 3 LEADER]: Becoming Leader. State: Replica: 46b559b325994b509ce1b821e583156a, State: Running, Role: LEADER
I20260925 05:15:34.949409 21373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } }
I20260925 05:15:34.949759 21376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46b559b325994b509ce1b821e583156a. Latest consensus state: current_term: 3 leader_uuid: "46b559b325994b509ce1b821e583156a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:34.949867 21376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:34.950078 21378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:34.950495 21378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:34.950579 21378 catalog_manager.cc:1259] Loaded cluster ID: dcf272ed2e844283b2bb5dd57c20aa84
I20260925 05:15:34.950613 21378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:34.950798 21378 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:34.950878 21378 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Loaded TSK: 0
I20260925 05:15:34.951050 21378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:35.036414 20644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:35.038105 20644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:35.039343 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.039209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42702)
0925 05:15:35.039274 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.039307 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 05:15:35.039350 20846 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [term 3 FOLLOWER]: Refusing update from remote peer 46b559b325994b509ce1b821e583156a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 05:15:35.039636 21376 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [LEADER]: Connected to new peer: Peer: permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 05:15:35.039714 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.039639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42710)
0925 05:15:35.039667 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.039690 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 05:15:35.039824 20630 consensus_peers.cc:487] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a -> Peer a2b329fde8734929a8cf5d69c3fdc2e3 (127.13.160.190:44123): Couldn't send request to peer a2b329fde8734929a8cf5d69c3fdc2e3. Status: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 05:15:35.040303 21380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46b559b325994b509ce1b821e583156a. Latest consensus state: current_term: 3 leader_uuid: "46b559b325994b509ce1b821e583156a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:35.040423 21380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:15:35.040808 21379 mvcc.cc:205] Tried to move back new op lower bound from 7333123420318597120 to 7333123419953442816. Current Snapshot: MvccSnapshot[applied={T|T < 7333123420318597120}]
I20260925 05:15:35.040906 21373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "46b559b325994b509ce1b821e583156a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:35.040998 21373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:35.041018 21380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "46b559b325994b509ce1b821e583156a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "a2b329fde8734929a8cf5d69c3fdc2e3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 44123 } } peers { permanent_uuid: "46b559b325994b509ce1b821e583156a" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46643 } attrs { promote: false } } peers { permanent_uuid: "991b79e8819b451ba3665c30a72cd897" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 33765 } attrs { promote: false } } }
I20260925 05:15:35.041124 21380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991b79e8819b451ba3665c30a72cd897 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:15:35.046439 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44139 (local address 127.0.0.1:34774)
0925 05:15:35.043070 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.043081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:35.043132 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:35.043566 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:35.043567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:35.043568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:35.043593 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.043594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044514 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.044515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.045531 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.045533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046207 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.046272 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046277 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:35.046306 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 05:15:35.046351 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":85}
I20260925 05:15:35.046799 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.13.160.129:44139)
0925 05:15:35.043437 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:35.043439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:35.043447 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:35.043472 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:35.043472 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:35.043482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:35.043672 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:35.043710 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044449 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.045624 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046110 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046634 (+ 524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046645 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:35.046676 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 05:15:35.046726 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":59,"threads_started":1}
I20260925 05:15:35.046963 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39381 (local address 127.0.0.1:54784)
0925 05:15:35.043545 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.043555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:35.043615 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:35.043782 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:35.043783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:35.043784 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:35.043828 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.043829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044459 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.044460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.045876 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.045878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046680 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.046739 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046743 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:35.046787 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 05:15:35.046834 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260925 05:15:35.047102 20949 tablet_service.cc:1467] Processing CreateTablet for tablet 53db76601071401a859842c9da00f4b6 (DEFAULT_TABLE table=test-table [id=b83fa5a17d60402aa91b92546748187a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:35.047245 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53db76601071401a859842c9da00f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:35.047377 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.13.160.131:32777)
0925 05:15:35.043329 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:35.043332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:35.043344 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:15:35.043376 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:35.043376 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:35.043388 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:35.043467 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:35.043662 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044449 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.045924 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046626 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046691 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046698 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:35.047216 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 05:15:35.047294 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260925 05:15:35.048614 21391 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: Bootstrap starting.
I20260925 05:15:35.048759 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54784 (local address 127.13.160.130:39381)
0925 05:15:35.043499 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:35.043501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:35.043669 (+ 168us) server_negotiation.cc:407] Connection header received
0925 05:15:35.043698 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:35.043699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:35.043709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:35.043766 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:35.043876 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044372 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.045962 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046527 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046603 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046616 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:35.048633 (+ 2017us) server_negotiation.cc:299] Negotiation successful
0925 05:15:35.048694 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260925 05:15:35.047055 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.043129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:32777 (local address 127.0.0.1:54792)
0925 05:15:35.043258 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.043270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:35.043323 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:35.043523 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:35.043524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:35.043527 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:35.043584 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.043586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.044604 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.044604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.045832 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:35.045835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.046729 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.046730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:35.046789 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.046793 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:35.046841 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 05:15:35.046917 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":11,"thread_start_us":83,"threads_started":1}
I20260925 05:15:35.049105 21222 tablet_service.cc:1467] Processing CreateTablet for tablet 53db76601071401a859842c9da00f4b6 (DEFAULT_TABLE table=test-table [id=b83fa5a17d60402aa91b92546748187a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:35.049108 21086 tablet_service.cc:1467] Processing CreateTablet for tablet 53db76601071401a859842c9da00f4b6 (DEFAULT_TABLE table=test-table [id=b83fa5a17d60402aa91b92546748187a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:35.049239 21086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53db76601071401a859842c9da00f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:35.049247 21222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53db76601071401a859842c9da00f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:35.050303 21393 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Bootstrap starting.
I20260925 05:15:35.050315 21392 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Bootstrap starting.
I20260925 05:15:35.050374 21391 tablet_bootstrap.cc:654] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:35.050701 21392 tablet_bootstrap.cc:654] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:35.050869 21393 tablet_bootstrap.cc:654] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:35.051118 21391 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: No bootstrap required, opened a new log
I20260925 05:15:35.051185 21391 ts_tablet_manager.cc:1397] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 05:15:35.051339 21392 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: No bootstrap required, opened a new log
I20260925 05:15:35.051394 21392 ts_tablet_manager.cc:1397] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:35.051348 21391 raft_consensus.cc:348] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051424 21391 raft_consensus.cc:374] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:35.051452 21391 raft_consensus.cc:729] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a917e3c1a54f388d980e986957a79e, State: Initialized, Role: FOLLOWER
I20260925 05:15:35.051450 21393 tablet_bootstrap.cc:492] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: No bootstrap required, opened a new log
I20260925 05:15:35.051491 21393 ts_tablet_manager.cc:1397] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:35.051501 21391 consensus_queue.cc:260] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051513 21392 raft_consensus.cc:348] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051591 21392 raft_consensus.cc:374] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:35.051635 21392 raft_consensus.cc:729] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b9d139d2df4f3584173c1c04608085, State: Initialized, Role: FOLLOWER
I20260925 05:15:35.051594 21393 raft_consensus.cc:348] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051678 21393 raft_consensus.cc:374] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:35.051697 21391 ts_tablet_manager.cc:1428] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:35.051716 21393 raft_consensus.cc:729] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, State: Initialized, Role: FOLLOWER
I20260925 05:15:35.051687 21392 consensus_queue.cc:260] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051787 21393 consensus_queue.cc:260] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.051880 21392 ts_tablet_manager.cc:1428] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:35.051955 21393 ts_tablet_manager.cc:1428] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:35.052037 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.051937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:44207)
0925 05:15:35.051981 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.052011 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 05:15:35.052474 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.052370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:51231)
0925 05:15:35.052411 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.052445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 05:15:35.052709 21017 heartbeater.cc:503] Master 127.13.160.189:46643 was elected leader, sending a full tablet report...
I20260925 05:15:35.052835 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.052755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:58575)
0925 05:15:35.052789 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.052812 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 05:15:35.052862 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.052761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:60593)
0925 05:15:35.052804 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.052831 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 05:15:35.052945 21290 heartbeater.cc:503] Master 127.13.160.189:46643 was elected leader, sending a full tablet report...
I20260925 05:15:35.053020 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.052930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:52091)
0925 05:15:35.052966 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.052991 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 05:15:35.053325 21154 heartbeater.cc:503] Master 127.13.160.189:46643 was elected leader, sending a full tablet report...
I20260925 05:15:35.053511 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.053376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:33853)
0925 05:15:35.053460 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.053485 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 05:15:35.393865 21399 raft_consensus.cc:482] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:35.393987 21399 raft_consensus.cc:504] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.394193 21399 leader_election.cc:288] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139), 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:35.394455 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53db76601071401a859842c9da00f4b6" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b9d139d2df4f3584173c1c04608085" is_pre_election: true
I20260925 05:15:35.394559 21242 raft_consensus.cc:2454] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 0.
I20260925 05:15:35.394482 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53db76601071401a859842c9da00f4b6" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a917e3c1a54f388d980e986957a79e" is_pre_election: true
I20260925 05:15:35.394613 20969 raft_consensus.cc:2454] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 0.
I20260925 05:15:35.394779 21039 leader_election.cc:302] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, 95a917e3c1a54f388d980e986957a79e; no voters:
I20260925 05:15:35.394927 21399 raft_consensus.cc:2781] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:35.394980 21399 raft_consensus.cc:482] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:35.395004 21399 raft_consensus.cc:3037] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:35.395784 21399 raft_consensus.cc:504] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.395939 21399 leader_election.cc:288] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 election: Requested vote from peers 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139), 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777)
I20260925 05:15:35.396108 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53db76601071401a859842c9da00f4b6" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a917e3c1a54f388d980e986957a79e"
I20260925 05:15:35.396186 20969 raft_consensus.cc:3037] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:35.396150 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53db76601071401a859842c9da00f4b6" candidate_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b9d139d2df4f3584173c1c04608085"
I20260925 05:15:35.396261 21242 raft_consensus.cc:3037] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:35.396843 20969 raft_consensus.cc:2454] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 1.
I20260925 05:15:35.397032 21242 raft_consensus.cc:2454] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c19be2fdd5f4eb19c0e2af73d4c6ea2 in term 1.
I20260925 05:15:35.397047 21039 leader_election.cc:302] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, 95a917e3c1a54f388d980e986957a79e; no voters:
I20260925 05:15:35.397146 21399 raft_consensus.cc:2781] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:35.397317 21399 raft_consensus.cc:686] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 5c19be2fdd5f4eb19c0e2af73d4c6ea2, State: Running, Role: LEADER
I20260925 05:15:35.397364 21399 consensus_queue.cc:237] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } }
I20260925 05:15:35.397878 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.397676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:42467)
0925 05:15:35.397785 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.397817 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260925 05:15:35.398196 20644 catalog_manager.cc:5495] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c19be2fdd5f4eb19c0e2af73d4c6ea2 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c19be2fdd5f4eb19c0e2af73d4c6ea2" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 39381 } health_report { overall_health: HEALTHY } } }
I20260925 05:15:35.440793 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.440641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42714)
0925 05:15:35.440706 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.440752 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 05:15:35.441228 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.441125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.0.0.1:42724)
0925 05:15:35.441173 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.441206 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 05:15:35.487344 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.485168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.13.160.130:39381)
0925 05:15:35.485232 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:35.485233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:35.485268 (+ 35us) server_negotiation.cc:407] Connection header received
0925 05:15:35.485292 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:35.485293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:35.485302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:35.485342 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:35.485471 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.485958 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.486672 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:35.486859 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:35.486965 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:35.487085 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:35.487112 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:35.487153 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:35.487208 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 05:15:35.487228 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:15:35.488349 21242 raft_consensus.cc:1261] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Refusing update from remote peer 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:35.488358 20969 raft_consensus.cc:1261] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Refusing update from remote peer 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:35.488597 21400 consensus_queue.cc:1035] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b9d139d2df4f3584173c1c04608085" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:35.488716 21399 consensus_queue.cc:1035] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a917e3c1a54f388d980e986957a79e" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:35.489645 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.489477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:45821)
0925 05:15:35.489570 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.489605 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 05:15:35.489787 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.489479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:57139)
0925 05:15:35.489730 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.489759 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260925 05:15:35.490819 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.490719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:54091)
0925 05:15:35.490760 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.490786 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 05:15:35.491153 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.491073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.130:47037)
0925 05:15:35.491103 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.491125 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 05:15:35.491384 21405 mvcc.cc:205] Tried to move back new op lower bound from 7333123422158393344 to 7333123421788024832. Current Snapshot: MvccSnapshot[applied={T|T < 7333123422158393344}]
I20260925 05:15:35.491963 20643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:35.492010 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.491873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.129:39389)
0925 05:15:35.491955 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.491983 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 05:15:35.492056 20643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:35.492142 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.492043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:44123 (local address 127.13.160.131:42647)
0925 05:15:35.492082 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.492114 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:44123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 05:15:35.492513 21403 mvcc.cc:205] Tried to move back new op lower bound from 7333123422158393344 to 7333123421788024832. Current Snapshot: MvccSnapshot[applied={T|T < 7333123422158393344 or (T in {7333123422158393344})}]
I20260925 05:15:35.493458 20643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46b559b325994b509ce1b821e583156a: Sending DeleteTablet for 3 replicas of tablet 53db76601071401a859842c9da00f4b6
I20260925 05:15:35.493796 21086 tablet_service.cc:1514] Processing DeleteTablet for tablet 53db76601071401a859842c9da00f4b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:35 UTC) from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:54784
I20260925 05:15:35.493846 21222 tablet_service.cc:1514] Processing DeleteTablet for tablet 53db76601071401a859842c9da00f4b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:35 UTC) from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:54792
I20260925 05:15:35.493901 21221 tablet_service.cc:1514] Processing DeleteTablet for tablet 53db76601071401a859842c9da00f4b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:35 UTC) from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:54792
I20260925 05:15:35.494068 20949 tablet_service.cc:1514] Processing DeleteTablet for tablet 53db76601071401a859842c9da00f4b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:35 UTC) from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:34774
I20260925 05:15:35.494131 21412 tablet_replica.cc:330] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: stopping tablet replica
I20260925 05:15:35.494181 21412 raft_consensus.cc:2229] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:35.494225 21412 raft_consensus.cc:2258] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:35.494253 20948 tablet_service.cc:1514] Processing DeleteTablet for tablet 53db76601071401a859842c9da00f4b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:35 UTC) from {username='oryx', principal='oryx/127.13.160.189@KRBTEST.COM'} at 127.0.0.1:34774
I20260925 05:15:35.494344 21413 tablet_replica.cc:330] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: stopping tablet replica
I20260925 05:15:35.494401 21413 raft_consensus.cc:2229] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:35.494414 21411 tablet_replica.cc:330] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: stopping tablet replica
I20260925 05:15:35.494442 21413 raft_consensus.cc:2258] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:35.494457 21411 raft_consensus.cc:2229] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:35.494513 21411 raft_consensus.cc:2258] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:35.494590 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20791
I20260925 05:15:35.494729 21413 ts_tablet_manager.cc:1905] T 53db76601071401a859842c9da00f4b6 P 95a917e3c1a54f388d980e986957a79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:35.494472 21412 ts_tablet_manager.cc:1905] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:35.497174 21412 ts_tablet_manager.cc:1918] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:35.497249 21412 log.cc:1196] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/53db76601071401a859842c9da00f4b6
I20260925 05:15:35.497473 21412 ts_tablet_manager.cc:1939] T 53db76601071401a859842c9da00f4b6 P 57b9d139d2df4f3584173c1c04608085: Deleting consensus metadata
I20260925 05:15:35.497828 20628 catalog_manager.cc:4841] TS 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777): tablet 53db76601071401a859842c9da00f4b6 (table test-table [id=b83fa5a17d60402aa91b92546748187a]) successfully deleted
W20260925 05:15:35.498016 20628 catalog_manager.cc:4808] TS 57b9d139d2df4f3584173c1c04608085 (127.13.160.131:32777): delete failed for tablet 53db76601071401a859842c9da00f4b6 because the tablet was not found. No further retry: Not found: Tablet not found: 53db76601071401a859842c9da00f4b6
I20260925 05:15:35.499764 21411 ts_tablet_manager.cc:1905] T 53db76601071401a859842c9da00f4b6 P 5c19be2fdd5f4eb19c0e2af73d4c6ea2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 05:15:35.500734 21387 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:35.500577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44139 (local address 127.0.0.1:34786)
0925 05:15:35.500633 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.500645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:35.500695 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:44139: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 05:15:35.500886 20628 catalog_manager.cc:4584] TS 95a917e3c1a54f388d980e986957a79e (127.13.160.129:44139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53db76601071401a859842c9da00f4b6: Network error: Client connection negotiation failed: client connection to 127.13.160.129:44139: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260925 05:15:35.501048 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21023
I20260925 05:15:35.507812 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:35.507670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:39381 (local address 127.0.0.1:54796)
0925 05:15:35.507744 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:35.507774 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:39381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 05:15:35.507936 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21159
I20260925 05:15:35.512729 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20509
I20260925 05:15:35.517685 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 20698
2026-09-25T05:15:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6907 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:35 dist-test-slave-9tgz krb5kdc[21416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:35 dist-test-slave-9tgz krb5kdc[21416](info): set up 2 sockets
Sep 25 05:15:35 dist-test-slave-9tgz krb5kdc[21416](info): commencing operation
krb5kdc: starting...
W20260925 05:15:36.556241 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:36 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab.
Entry for principal oryx/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab.
I20260925 05:15:36.680143 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35197
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab
--principal=oryx/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:35197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:36.763504 21429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:36.763732 21429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:36.763756 21429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:36.765065 21429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:36.765116 21429 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 05:15:36.765136 21429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:36.765148 21429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:36.765161 21429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:36.765173 21429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:36.766814 21429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:35197
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.190.keytab
--principal=oryx/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:36.767145 21429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:36.767385 21429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:36.772588 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:36.772607 21429 server_base.cc:1034] running on GCE node
W20260925 05:15:36.772588 21437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:36.772598 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:36.773106 21429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:36.773449 21429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:36.774621 21429 hybrid_clock.cc:648] HybridClock initialized: now 1790313336774613 us; error 38 us; skew 500 ppm
Sep 25 05:15:36 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:36.777207 21429 init.cc:377] Logged in from keytab as oryx/127.13.160.190@KRBTEST.COM (short username oryx)
I20260925 05:15:36.777642 21429 webserver.cc:460] Webserver started at http://127.13.160.190:41021/ using document root <none> and password file <none>
I20260925 05:15:36.777892 21429 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:36.777940 21429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:36.778034 21429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:36.779314 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "006faed066344638a413550eeeefd8a2"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "47ac4410b26fba6dd42d95128a09c464"
server_key_iv: "85d88d240257ff939caabb2befb676a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.779649 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "006faed066344638a413550eeeefd8a2"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "47ac4410b26fba6dd42d95128a09c464"
server_key_iv: "85d88d240257ff939caabb2befb676a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.780938 21429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:36.781636 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:36.781831 21429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:36.781917 21429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "006faed066344638a413550eeeefd8a2"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "47ac4410b26fba6dd42d95128a09c464"
server_key_iv: "85d88d240257ff939caabb2befb676a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.781975 21429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:36.799297 21429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:36.801260 21429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:36.801378 21429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:36.806313 21429 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:35197
I20260925 05:15:36.806396 21506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:35197 every 8 connection(s)
I20260925 05:15:36.806759 21429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:36.807535 21507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:36.809908 21507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Bootstrap starting.
I20260925 05:15:36.810535 21507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:36.810850 21507 log.cc:824] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:36.811646 21507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: No bootstrap required, opened a new log
I20260925 05:15:36.813742 21507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } }
I20260925 05:15:36.813915 21507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:36.813962 21507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006faed066344638a413550eeeefd8a2, State: Initialized, Role: FOLLOWER
I20260925 05:15:36.814145 21507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } }
I20260925 05:15:36.814248 21507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:36.814293 21507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:36.814342 21507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:36.814967 21507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } }
I20260925 05:15:36.815107 21507 leader_election.cc:302] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 006faed066344638a413550eeeefd8a2; no voters:
I20260925 05:15:36.815285 21507 leader_election.cc:288] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:36.815343 21510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:36.815589 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21429
I20260925 05:15:36.815691 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:36.815861 13954 external_mini_cluster.cc:1428] Setting key 6d866e3a98459047fe07bf38a023ee4e
I20260925 05:15:36.815565 21507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:36.816100 21510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [term 1 LEADER]: Becoming Leader. State: Replica: 006faed066344638a413550eeeefd8a2, State: Running, Role: LEADER
I20260925 05:15:36.816236 21510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } }
I20260925 05:15:36.816639 21512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006faed066344638a413550eeeefd8a2. Latest consensus state: current_term: 1 leader_uuid: "006faed066344638a413550eeeefd8a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } } }
I20260925 05:15:36.816722 21512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:36.816924 21511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "006faed066344638a413550eeeefd8a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006faed066344638a413550eeeefd8a2" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35197 } } }
I20260925 05:15:36.817026 21511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:36.817250 21521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:36.817770 21521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:36.819365 21521 catalog_manager.cc:1347] Generated new cluster ID: 3474c8242e9e4b9fb06d3a8bc887dff5
I20260925 05:15:36.819416 21521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:36 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:36.828460 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:36.816364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.13.160.190:35197)
0925 05:15:36.816860 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:36.816870 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:36.816906 (+ 36us) server_negotiation.cc:407] Connection header received
0925 05:15:36.817117 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:36.817126 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:36.817199 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:36.818194 (+ 995us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:36.818303 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:36.819695 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:36.820764 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:36.821427 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:36.821547 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:36.824435 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:36.824441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:36.824441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:36.824455 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:36.826537 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:36.827003 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:36.827007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:36.827008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:36.827132 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:36.827353 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:36.827354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:36.827355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:36.827710 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:36.827875 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:36.827949 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 05:15:36.828210 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":65,"threads_started":1}
I20260925 05:15:36.835263 21521 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:36.836069 21521 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:36.852219 21521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Generated new TSK 0
I20260925 05:15:36.852481 21521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
I20260925 05:15:36.878212 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35197
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:36.962765 21533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:36.963070 21533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:36.963102 21533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:36.964337 21533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:36.964386 21533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:36.964407 21533 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:36.964450 21533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:36.966073 21533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:36.966368 21533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:36.966616 21533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:36.971686 21538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:36.971845 21539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:36.971899 21541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:36.971935 21533 server_base.cc:1034] running on GCE node
I20260925 05:15:36.972159 21533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:36.972443 21533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:36.973619 21533 hybrid_clock.cc:648] HybridClock initialized: now 1790313336973604 us; error 43 us; skew 500 ppm
Sep 25 05:15:36 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:36.976233 21533 init.cc:377] Logged in from keytab as oryx/127.13.160.129@KRBTEST.COM (short username oryx)
I20260925 05:15:36.976722 21533 webserver.cc:460] Webserver started at http://127.13.160.129:39181/ using document root <none> and password file <none>
I20260925 05:15:36.976966 21533 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:36.977034 21533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:36.977188 21533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:36.978329 21533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "8bffbe9e621a4bfd855cf2265281f159"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "9ec6246dfd0459804e104efbd8243a7d"
server_key_iv: "383de132eced8f530acd597b926ab1f6"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.978703 21533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "8bffbe9e621a4bfd855cf2265281f159"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "9ec6246dfd0459804e104efbd8243a7d"
server_key_iv: "383de132eced8f530acd597b926ab1f6"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.980155 21533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:36.980885 21548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:36.981086 21533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:36.981186 21533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "8bffbe9e621a4bfd855cf2265281f159"
format_stamp: "Formatted at 2026-09-25 05:15:36 on dist-test-slave-9tgz"
server_key: "9ec6246dfd0459804e104efbd8243a7d"
server_key_iv: "383de132eced8f530acd597b926ab1f6"
server_key_version: "encryptionkey@0"
I20260925 05:15:36.981277 21533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:36.997090 21533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:36.999257 21533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:36.999435 21533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:37.000034 21533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:37.000097 21533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.000149 21533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:37.000167 21533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:37.007716 21533 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:45969
I20260925 05:15:37.007750 21663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:45969 every 8 connection(s)
I20260925 05:15:37.008164 21533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:37.010746 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.000470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:54111 (local address 127.13.160.190:35197)
0925 05:15:37.000548 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.000551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.000810 (+ 259us) server_negotiation.cc:407] Connection header received
0925 05:15:37.001318 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.001319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.001337 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.001419 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.002386 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.003147 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.004270 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.004495 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.004588 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.007356 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.007358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.007358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.007365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.009247 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.009712 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.009714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.009715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.009793 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.010066 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.010067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.010068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.010217 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.010290 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.010591 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.010676 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 05:15:37.010902 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.000480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.129:54111)
0925 05:15:37.000662 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.000704 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.001210 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.001516 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.001520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.001947 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.002270 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.002293 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.003257 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.003259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.004173 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.004176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.004614 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.004615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.004766 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.005450 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.005465 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.007240 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.009351 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.009360 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.009362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.009629 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.009869 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.009869 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.009870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.009982 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.010316 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.010322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.010462 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.010559 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.010700 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260925 05:15:37.011379 21664 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35197
I20260925 05:15:37.011461 21664 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:37.011654 21664 heartbeater.cc:511] Master 127.13.160.190:35197 requested a full tablet report, sending...
I20260925 05:15:37.012228 21460 ts_manager.cc:194] Registered new tserver with Master: 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969)
I20260925 05:15:37.013098 21460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:54111
I20260925 05:15:37.013700 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21533
I20260925 05:15:37.013808 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:37.014005 13954 external_mini_cluster.cc:1428] Setting key b4ec0e47d72e73aa643a64d1f20e1057
I20260925 05:15:37.018594 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.012462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.129:33179)
0925 05:15:37.012619 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.012630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.012680 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.012794 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.012795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.012953 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.013012 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.013014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.013761 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.013762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.014383 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.014386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.014789 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.014790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.014862 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.015321 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.015326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.015758 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.017670 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.017670 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.017671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.017836 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.018022 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.018022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.018022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.018073 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.018352 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.018352 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.018392 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.018443 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.018522 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260925 05:15:37.018725 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.012543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33179 (local address 127.13.160.190:35197)
0925 05:15:37.012584 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.012586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.012687 (+ 101us) server_negotiation.cc:407] Connection header received
0925 05:15:37.012715 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.012715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.012725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.012791 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.013074 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.013647 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.014464 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.014697 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.014775 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.015837 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.015838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.015838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.015844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.017589 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.017892 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.017893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.017894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.017957 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.018132 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.018133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.018133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.018281 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.018357 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.018488 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.018584 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
I20260925 05:15:37.039920 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35197
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:37.124018 21673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:37.124249 21673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:37.124284 21673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:37.125458 21673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:37.125501 21673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:37.125522 21673 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:37.125540 21673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:37.127089 21673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:37.127311 21673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:37.127502 21673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:37.132721 21678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:37.132721 21679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:37.132776 21681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:37.132704 21673 server_base.cc:1034] running on GCE node
I20260925 05:15:37.133191 21673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:37.133486 21673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:37.134666 21673 hybrid_clock.cc:648] HybridClock initialized: now 1790313337134660 us; error 39 us; skew 500 ppm
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:37.137318 21673 init.cc:377] Logged in from keytab as oryx/127.13.160.130@KRBTEST.COM (short username oryx)
I20260925 05:15:37.137746 21673 webserver.cc:460] Webserver started at http://127.13.160.130:40207/ using document root <none> and password file <none>
I20260925 05:15:37.137953 21673 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:37.137996 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:37.138090 21673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:37.139241 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "360d959122294219a6abbb5fac2d88fb"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "ec527fa0d36526644a2db5c7574eaa65"
server_key_iv: "4f56e096a47638b6934508f3a54969c3"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.139559 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "360d959122294219a6abbb5fac2d88fb"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "ec527fa0d36526644a2db5c7574eaa65"
server_key_iv: "4f56e096a47638b6934508f3a54969c3"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.140868 21673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:37.141530 21688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.141688 21673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:37.141774 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "360d959122294219a6abbb5fac2d88fb"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "ec527fa0d36526644a2db5c7574eaa65"
server_key_iv: "4f56e096a47638b6934508f3a54969c3"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.141831 21673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:37.162384 21673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:37.164908 21673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:37.165035 21673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:37.165629 21673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:37.165678 21673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.165702 21673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:37.165715 21673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.172559 21673 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:40903
I20260925 05:15:37.172670 21803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:40903 every 8 connection(s)
I20260925 05:15:37.173018 21673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:37.175419 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21673
I20260925 05:15:37.175544 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:37.175752 13954 external_mini_cluster.cc:1428] Setting key c678558af94f0c4e60079fed7d64804f
I20260925 05:15:37.177249 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.166206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:54901 (local address 127.13.160.190:35197)
0925 05:15:37.166289 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.166291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.166472 (+ 181us) server_negotiation.cc:407] Connection header received
0925 05:15:37.167087 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.167088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.167105 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.167190 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.168185 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.169038 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.170684 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.171051 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.171145 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.174109 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.174110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.174110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.174117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.175687 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.176208 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.176209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.176209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.176261 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.176549 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.176549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.176550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.176710 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.176786 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.177070 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.177157 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 05:15:37.177472 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.166132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.130:54901)
0925 05:15:37.166330 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.166373 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.166942 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.167296 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.167301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.167717 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.168048 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.168055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.169164 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.169165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.170575 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.170579 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.171159 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.171160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.171312 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.172026 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.172043 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.174002 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.175769 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.175775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.175777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.176138 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.176328 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.176328 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.176329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.176476 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.176783 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.176788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.176911 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.177042 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.177210 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260925 05:15:37.178956 21804 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35197
I20260925 05:15:37.179054 21804 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:37.179232 21804 heartbeater.cc:511] Master 127.13.160.190:35197 requested a full tablet report, sending...
I20260925 05:15:37.179571 21460 ts_manager.cc:194] Registered new tserver with Master: 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903)
I20260925 05:15:37.180090 21460 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:54901
WARNING: no policy specified for oryx/127.13.160.131@KRBTEST.COM; defaulting to no policy
I20260925 05:15:37.185627 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.179113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:57045 (local address 127.13.160.190:35197)
0925 05:15:37.179192 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.179194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.179312 (+ 118us) server_negotiation.cc:407] Connection header received
0925 05:15:37.179390 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.179391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.179401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.179449 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.179888 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.180466 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.181473 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.181678 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.181751 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.182750 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.182752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.182752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.182760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.184333 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.184683 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.184684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.184685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.184740 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.184964 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.184965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.184966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.185124 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.185210 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.185441 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.185511 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 05:15:37.185652 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.179091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.130:57045)
0925 05:15:37.179246 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.179256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.179336 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.179524 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.179525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.179725 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.179796 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.179798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.180607 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.180608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.181376 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.181378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.181749 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.181750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.181821 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.182333 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.182339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.182641 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.184413 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.184414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.184414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.184594 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.184825 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.184826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.184827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.184891 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.185253 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.185254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.185315 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.185395 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.185580 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
I20260925 05:15:37.201771 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35197
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:37.288295 21813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:37.288681 21813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:37.288789 21813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:37.290411 21813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:37.290483 21813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:37.290518 21813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 05:15:37.290556 21813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:37.292184 21813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.160.190:35197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:37.292508 21813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:37.292770 21813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:37.297993 21818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:37.297988 21821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:37.297989 21819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:37.298206 21813 server_base.cc:1034] running on GCE node
I20260925 05:15:37.298493 21813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:37.298837 21813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:37.300034 21813 hybrid_clock.cc:648] HybridClock initialized: now 1790313337300034 us; error 47 us; skew 500 ppm
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:37.302620 21813 init.cc:377] Logged in from keytab as oryx/127.13.160.131@KRBTEST.COM (short username oryx)
I20260925 05:15:37.303169 21813 webserver.cc:460] Webserver started at http://127.13.160.131:36487/ using document root <none> and password file <none>
I20260925 05:15:37.303412 21813 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:37.303460 21813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:37.303609 21813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:37.304719 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "713be8f34819496a8c0c4ae262c6cce8"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "e9636b37a9dcb500f593ca88d7d64f05"
server_key_iv: "6f4255b0a1a163674cb878a92f064ecb"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.305083 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "713be8f34819496a8c0c4ae262c6cce8"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "e9636b37a9dcb500f593ca88d7d64f05"
server_key_iv: "6f4255b0a1a163674cb878a92f064ecb"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.306468 21813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:37.307307 21828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.307484 21813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.307591 21813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "713be8f34819496a8c0c4ae262c6cce8"
format_stamp: "Formatted at 2026-09-25 05:15:37 on dist-test-slave-9tgz"
server_key: "e9636b37a9dcb500f593ca88d7d64f05"
server_key_iv: "6f4255b0a1a163674cb878a92f064ecb"
server_key_version: "encryptionkey@0"
I20260925 05:15:37.307672 21813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:37.329734 21813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:37.331912 21813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:37.332104 21813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:37.332700 21813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:37.332768 21813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.332814 21813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:37.332842 21813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:37.340276 21813 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:46875
I20260925 05:15:37.340371 21943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:46875 every 8 connection(s)
I20260925 05:15:37.340672 21813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:37.342720 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.333191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:55563 (local address 127.13.160.190:35197)
0925 05:15:37.333264 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.333266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.333443 (+ 177us) server_negotiation.cc:407] Connection header received
0925 05:15:37.333965 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.333966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.333982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.334064 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.335076 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.335591 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.336644 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.336914 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.336996 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.339878 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.339880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.339880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.339888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.341549 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.341924 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.341925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.341925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.341979 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.342195 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.342195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.342196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.342303 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.342353 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.342585 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.342656 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 05:15:37.342941 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.333168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.131:55563)
0925 05:15:37.333312 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.333354 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.333864 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.334178 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.334183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.334582 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.334962 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.334969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.335690 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.335690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.336540 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.336541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.337007 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.337008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.337174 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.337789 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.337800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.339753 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.341631 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.341635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.341637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.341854 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.342041 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.342042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.342042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.342141 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.342357 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.342361 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.342476 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.342559 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.342706 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260925 05:15:37.343430 21944 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35197
I20260925 05:15:37.343528 21944 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:37.343712 21944 heartbeater.cc:511] Master 127.13.160.190:35197 requested a full tablet report, sending...
I20260925 05:15:37.344115 21461 ts_manager.cc:194] Registered new tserver with Master: 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:37.344589 21461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:55563
I20260925 05:15:37.347342 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 21813
I20260925 05:15:37.347432 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:37.347641 13954 external_mini_cluster.cc:1428] Setting key c349411d83f69f2adfb9e0a2fdfc652f
I20260925 05:15:37.348870 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:37.350381 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.344511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:56117 (local address 127.13.160.190:35197)
0925 05:15:37.344550 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.344552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.344656 (+ 104us) server_negotiation.cc:407] Connection header received
0925 05:15:37.344775 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.344776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.344786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.344836 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.345169 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.345687 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.346372 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.346586 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.346680 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.347676 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.347678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.347678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.347685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.349216 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.349556 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.349557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.349557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.349612 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.349824 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.349825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.349825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.349967 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.350040 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.350203 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.350283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:37.350407 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.344453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.13.160.131:56117)
0925 05:15:37.344573 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.344585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.344651 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.344844 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.344845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.345005 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.345070 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.345109 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.345778 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.345779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.346289 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.346291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.346682 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.346683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.346747 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.347221 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.347225 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.347577 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.349294 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.349295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.349295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.349469 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.349685 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.349685 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.349686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.349743 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.350042 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.350043 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.350091 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.350180 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.350284 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 05:15:37.370800 21515 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:37.365257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.13.160.190:35197)
0925 05:15:37.365335 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.365337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.365346 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:37.365411 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.365412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.365421 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.365473 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.366037 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.366539 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.367349 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.367532 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.367589 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.370683 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50668: BlockingRecv error: recv got EOF from 127.0.0.1:50668 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:37.382169 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.371691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.13.160.190:35197)
0925 05:15:37.371748 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.371750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.371759 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:37.371809 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.371810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.371818 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.371871 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.372559 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.373463 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.374647 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.375018 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.375087 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.378964 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.378966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.378966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.378975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.380837 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.381317 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.381319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.381319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.381397 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.381589 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.381591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.381591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.381804 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.381913 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.382035 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.382098 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 05:15:37.384044 21461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:37.385936 21461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:37.392745 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:46875 (local address 127.0.0.1:35704)
0925 05:15:37.388490 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.388534 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.388606 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.389310 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.389312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.389315 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.389358 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.389359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390253 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.390254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.391774 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.391776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392463 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.392514 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.392524 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.392571 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.392625 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260925 05:15:37.392781 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.13.160.129:45969)
0925 05:15:37.388967 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.388972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.389370 (+ 398us) server_negotiation.cc:407] Connection header received
0925 05:15:37.389412 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.389414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.389427 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.389502 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.389587 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390563 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.391578 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392331 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392426 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.392484 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.392624 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.392674 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260925 05:15:37.392745 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:45969 (local address 127.0.0.1:40478)
0925 05:15:37.388957 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.388967 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.389014 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.389498 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.389499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.389500 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.389530 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.389531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390651 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.390652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.391467 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.391469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392426 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.392506 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.392521 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.392598 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.392638 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":4,"thread_start_us":62,"threads_started":1}
I20260925 05:15:37.392939 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:40903 (local address 127.0.0.1:43084)
0925 05:15:37.388741 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.388751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.388811 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.389044 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.389047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.389050 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.389127 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.389129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390411 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.390412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.391573 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.391576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392524 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.392628 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.392632 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.392685 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.392731 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":30,"thread_start_us":90,"threads_started":1}
I20260925 05:15:37.392959 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.13.160.130:40903)
0925 05:15:37.388676 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.388680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.388878 (+ 198us) server_negotiation.cc:407] Connection header received
0925 05:15:37.388925 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.388928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.388945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.389031 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.389209 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390329 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.391641 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392331 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392426 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.392484 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.392802 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.392845 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260925 05:15:37.393778 21598 tablet_service.cc:1467] Processing CreateTablet for tablet 59a79f6670214271af2372ba2213ef97 (DEFAULT_TABLE table=test-table [id=8637d58cb8654af48c1c8a24454bb401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.393780 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 59a79f6670214271af2372ba2213ef97 (DEFAULT_TABLE table=test-table [id=8637d58cb8654af48c1c8a24454bb401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.394150 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a79f6670214271af2372ba2213ef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.394150 21598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a79f6670214271af2372ba2213ef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.395143 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.388535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.13.160.131:46875)
0925 05:15:37.389145 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.389149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.389160 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:37.389218 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.389219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.389233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.389313 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.389436 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.390165 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.391862 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.392349 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.392807 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.393366 (+ 559us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.394987 (+ 1621us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.395047 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":81,"threads_started":1}
I20260925 05:15:37.397481 21972 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Bootstrap starting.
I20260925 05:15:37.397644 21973 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Bootstrap starting.
I20260925 05:15:37.398205 21972 tablet_bootstrap.cc:654] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.398293 21973 tablet_bootstrap.cc:654] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.398315 21878 tablet_service.cc:1467] Processing CreateTablet for tablet 59a79f6670214271af2372ba2213ef97 (DEFAULT_TABLE table=test-table [id=8637d58cb8654af48c1c8a24454bb401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.398536 21972 log.cc:824] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:37.398607 21973 log.cc:824] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:37.398705 21878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a79f6670214271af2372ba2213ef97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.399382 21973 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: No bootstrap required, opened a new log
I20260925 05:15:37.399415 21972 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: No bootstrap required, opened a new log
I20260925 05:15:37.399470 21972 ts_tablet_manager.cc:1397] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:37.399468 21973 ts_tablet_manager.cc:1397] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:37.401069 21973 raft_consensus.cc:348] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.401227 21973 raft_consensus.cc:374] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.401270 21973 raft_consensus.cc:729] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.401381 21973 consensus_queue.cc:260] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.401678 21973 ts_tablet_manager.cc:1428] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:37.401683 21804 heartbeater.cc:503] Master 127.13.160.190:35197 was elected leader, sending a full tablet report...
I20260925 05:15:37.401615 21972 raft_consensus.cc:348] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.401827 21972 raft_consensus.cc:374] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.401871 21972 raft_consensus.cc:729] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.401995 21972 consensus_queue.cc:260] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.402244 21972 ts_tablet_manager.cc:1428] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:15:37.402361 21664 heartbeater.cc:503] Master 127.13.160.190:35197 was elected leader, sending a full tablet report...
I20260925 05:15:37.402545 21978 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Bootstrap starting.
I20260925 05:15:37.403270 21978 tablet_bootstrap.cc:654] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.403573 21978 log.cc:824] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:37.404278 21978 tablet_bootstrap.cc:492] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: No bootstrap required, opened a new log
I20260925 05:15:37.404362 21978 ts_tablet_manager.cc:1397] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:37.405754 21978 raft_consensus.cc:348] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.405910 21978 raft_consensus.cc:374] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.405956 21978 raft_consensus.cc:729] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713be8f34819496a8c0c4ae262c6cce8, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.406076 21978 consensus_queue.cc:260] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.406352 21978 ts_tablet_manager.cc:1428] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:15:37.406375 21944 heartbeater.cc:503] Master 127.13.160.190:35197 was elected leader, sending a full tablet report...
W20260925 05:15:37.423664 21805 tablet.cc:2369] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:15:37.509240 21665 tablet.cc:2369] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:37.534819 21977 raft_consensus.cc:482] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:37.534986 21977 raft_consensus.cc:504] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.535466 21977 leader_election.cc:288] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875), 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903)
I20260925 05:15:37.538585 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.535763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:55817 (local address 127.13.160.130:40903)
0925 05:15:37.535819 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.535821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.535909 (+ 88us) server_negotiation.cc:407] Connection header received
0925 05:15:37.535983 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.535985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.536007 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.536061 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.536244 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.536762 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.537825 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.538215 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.538302 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.538311 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.538464 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.538502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 05:15:37.538623 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.535783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:59771 (local address 127.13.160.131:46875)
0925 05:15:37.535826 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.535829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.535944 (+ 115us) server_negotiation.cc:407] Connection header received
0925 05:15:37.536011 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.536012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.536027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.536102 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.536289 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.536762 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.537830 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.538250 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.538344 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.538352 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.538514 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.538552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:15:37.538633 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.535672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:40903 (local address 127.13.160.129:55817)
0925 05:15:37.535822 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.535834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.535901 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.536088 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.536089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.536091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.536172 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.536174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.536859 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.536860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.537727 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.537729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.538300 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.538301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.538374 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.538377 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.538455 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.538502 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":53,"thread_start_us":88,"threads_started":1}
I20260925 05:15:37.538652 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.535680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:46875 (local address 127.13.160.129:59771)
0925 05:15:37.535833 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.535844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.535939 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.536124 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.536124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.536127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.536191 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.536192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.536851 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.536853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.537722 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.537725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.538344 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.538345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.538426 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.538429 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.538505 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.538551 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":53,"thread_start_us":63,"threads_started":1}
I20260925 05:15:37.539170 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a79f6670214271af2372ba2213ef97" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb" is_pre_election: true
I20260925 05:15:37.539247 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a79f6670214271af2372ba2213ef97" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8" is_pre_election: true
I20260925 05:15:37.539364 21758 raft_consensus.cc:2454] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:37.539408 21898 raft_consensus.cc:2454] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:37.539644 21551 leader_election.cc:302] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:37.539783 21977 raft_consensus.cc:2781] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:37.539839 21977 raft_consensus.cc:482] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:37.539863 21977 raft_consensus.cc:3037] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.540642 21977 raft_consensus.cc:504] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.540795 21977 leader_election.cc:288] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Requested vote from peers 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875), 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903)
I20260925 05:15:37.540966 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a79f6670214271af2372ba2213ef97" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8"
I20260925 05:15:37.540977 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a79f6670214271af2372ba2213ef97" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb"
I20260925 05:15:37.541078 21898 raft_consensus.cc:3037] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.541075 21758 raft_consensus.cc:3037] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.541814 21898 raft_consensus.cc:2454] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:37.541849 21758 raft_consensus.cc:2454] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:37.542017 21551 leader_election.cc:302] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:37.542143 21977 raft_consensus.cc:2781] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:37.542276 21977 raft_consensus.cc:686] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Becoming Leader. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Running, Role: LEADER
I20260925 05:15:37.542390 21977 consensus_queue.cc:237] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } }
I20260925 05:15:37.543257 21460 catalog_manager.cc:5495] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "8bffbe9e621a4bfd855cf2265281f159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:37.573720 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.571436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.13.160.129:45969)
0925 05:15:37.571490 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.571492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.571500 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:37.571548 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.571549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.571568 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.571633 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:37.571793 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.572295 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.573028 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.573221 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.573269 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.573414 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.573525 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.573587 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.573636 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.573657 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 05:15:37.575655 21898 raft_consensus.cc:1261] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.575727 21758 raft_consensus.cc:1261] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.576056 21977 consensus_queue.cc:1035] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.576218 21983 consensus_queue.cc:1035] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.579557 21986 mvcc.cc:205] Tried to move back new op lower bound from 7333123430707113984 to 7333123430574292992. Current Snapshot: MvccSnapshot[applied={T|T < 7333123430707113984}]
I20260925 05:15:37.579671 21987 mvcc.cc:205] Tried to move back new op lower bound from 7333123430707113984 to 7333123430574292992. Current Snapshot: MvccSnapshot[applied={T|T < 7333123430707113984}]
I20260925 05:15:37.581743 21460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:37.581841 21460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:37.582655 21460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Sending DeleteTablet for 3 replicas of tablet 59a79f6670214271af2372ba2213ef97
I20260925 05:15:37.582984 21878 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a79f6670214271af2372ba2213ef97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:35704
I20260925 05:15:37.583034 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a79f6670214271af2372ba2213ef97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:43084
I20260925 05:15:37.583032 21598 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a79f6670214271af2372ba2213ef97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:40478
I20260925 05:15:37.583359 21997 tablet_replica.cc:330] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: stopping tablet replica
I20260925 05:15:37.583489 21995 tablet_replica.cc:330] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: stopping tablet replica
I20260925 05:15:37.583487 21460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:37.583654 21460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:37.583827 21997 raft_consensus.cc:2229] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:37.583941 21995 raft_consensus.cc:2229] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:37.583983 21996 tablet_replica.cc:330] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: stopping tablet replica
I20260925 05:15:37.584126 21996 raft_consensus.cc:2229] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:37.584249 21996 raft_consensus.cc:2258] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.584506 21995 raft_consensus.cc:2258] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.585052 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 (DEFAULT_TABLE table=test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.585179 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d10876dbc94529b09f1957b0bfbda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.585048 21598 tablet_service.cc:1467] Processing CreateTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 (DEFAULT_TABLE table=test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.585315 21996 ts_tablet_manager.cc:1905] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.585397 21598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d10876dbc94529b09f1957b0bfbda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.586604 21973 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Bootstrap starting.
I20260925 05:15:37.584666 21997 raft_consensus.cc:2258] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.585050 21878 tablet_service.cc:1467] Processing CreateTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 (DEFAULT_TABLE table=test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:37.586997 21996 ts_tablet_manager.cc:1918] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:37.587049 21878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d10876dbc94529b09f1957b0bfbda5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.587060 21996 log.cc:1196] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/59a79f6670214271af2372ba2213ef97
I20260925 05:15:37.587080 21973 tablet_bootstrap.cc:654] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.587366 21996 ts_tablet_manager.cc:1939] T 59a79f6670214271af2372ba2213ef97 P 360d959122294219a6abbb5fac2d88fb: Deleting consensus metadata
I20260925 05:15:37.587785 21447 catalog_manager.cc:4841] TS 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903): tablet 59a79f6670214271af2372ba2213ef97 (table test-table [id=8637d58cb8654af48c1c8a24454bb401]) successfully deleted
I20260925 05:15:37.588624 21978 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Bootstrap starting.
I20260925 05:15:37.587404 21972 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Bootstrap starting.
I20260925 05:15:37.589038 21978 tablet_bootstrap.cc:654] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.589226 21972 tablet_bootstrap.cc:654] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.589931 21972 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: No bootstrap required, opened a new log
I20260925 05:15:37.589969 21997 ts_tablet_manager.cc:1905] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.589985 21972 ts_tablet_manager.cc:1397] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 05:15:37.590164 21972 raft_consensus.cc:348] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.590240 21972 raft_consensus.cc:374] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.590263 21972 raft_consensus.cc:729] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.590308 21972 consensus_queue.cc:260] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.590395 21972 ts_tablet_manager.cc:1428] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.590780 21995 ts_tablet_manager.cc:1905] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.591614 21997 ts_tablet_manager.cc:1918] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:37.591709 21997 log.cc:1196] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/59a79f6670214271af2372ba2213ef97
I20260925 05:15:37.592029 21997 ts_tablet_manager.cc:1939] T 59a79f6670214271af2372ba2213ef97 P 8bffbe9e621a4bfd855cf2265281f159: Deleting consensus metadata
I20260925 05:15:37.592286 21995 ts_tablet_manager.cc:1918] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:37.592447 21995 log.cc:1196] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/59a79f6670214271af2372ba2213ef97
I20260925 05:15:37.592458 21448 catalog_manager.cc:4841] TS 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969): tablet 59a79f6670214271af2372ba2213ef97 (table test-table [id=8637d58cb8654af48c1c8a24454bb401]) successfully deleted
I20260925 05:15:37.592729 21995 ts_tablet_manager.cc:1939] T 59a79f6670214271af2372ba2213ef97 P 713be8f34819496a8c0c4ae262c6cce8: Deleting consensus metadata
I20260925 05:15:37.593139 21448 catalog_manager.cc:4841] TS 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875): tablet 59a79f6670214271af2372ba2213ef97 (table test-table [id=8637d58cb8654af48c1c8a24454bb401]) successfully deleted
I20260925 05:15:37.593390 21978 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: No bootstrap required, opened a new log
I20260925 05:15:37.593453 21978 ts_tablet_manager.cc:1397] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 05:15:37.593613 21978 raft_consensus.cc:348] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.593698 21978 raft_consensus.cc:374] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.593727 21978 raft_consensus.cc:729] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713be8f34819496a8c0c4ae262c6cce8, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.593775 21978 consensus_queue.cc:260] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.593863 21978 ts_tablet_manager.cc:1428] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.594242 21973 tablet_bootstrap.cc:492] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: No bootstrap required, opened a new log
I20260925 05:15:37.594298 21973 ts_tablet_manager.cc:1397] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260925 05:15:37.594465 21973 raft_consensus.cc:348] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.594540 21973 raft_consensus.cc:374] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.594563 21973 raft_consensus.cc:729] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.594616 21973 consensus_queue.cc:260] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.594714 21973 ts_tablet_manager.cc:1428] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.608207 21977 raft_consensus.cc:482] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:37.608328 21977 raft_consensus.cc:504] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.608493 21977 leader_election.cc:288] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:37.608701 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d10876dbc94529b09f1957b0bfbda5" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8" is_pre_election: true
I20260925 05:15:37.608697 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d10876dbc94529b09f1957b0bfbda5" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb" is_pre_election: true
I20260925 05:15:37.608796 21898 raft_consensus.cc:2454] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:37.608809 21758 raft_consensus.cc:2454] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:37.608963 21552 leader_election.cc:302] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713be8f34819496a8c0c4ae262c6cce8, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:37.609067 21977 raft_consensus.cc:2781] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:37.609117 21977 raft_consensus.cc:482] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:37.609138 21977 raft_consensus.cc:3037] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.610399 21977 raft_consensus.cc:504] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.610564 21977 leader_election.cc:288] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Requested vote from peers 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:37.610733 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d10876dbc94529b09f1957b0bfbda5" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb"
I20260925 05:15:37.610790 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d10876dbc94529b09f1957b0bfbda5" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8"
I20260925 05:15:37.610816 21758 raft_consensus.cc:3037] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.610867 21898 raft_consensus.cc:3037] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.611613 21758 raft_consensus.cc:2454] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:37.611692 21898 raft_consensus.cc:2454] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:37.611828 21551 leader_election.cc:302] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:37.611912 21977 raft_consensus.cc:2781] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:37.611974 21977 raft_consensus.cc:686] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Becoming Leader. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Running, Role: LEADER
I20260925 05:15:37.612025 21977 consensus_queue.cc:237] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.612603 21460 catalog_manager.cc:5495] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "8bffbe9e621a4bfd855cf2265281f159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } health_report { overall_health: UNKNOWN } } }
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313336, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.190@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:37.639810 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.631823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.13.160.190:35197)
0925 05:15:37.631898 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.631901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.632008 (+ 107us) server_negotiation.cc:407] Connection header received
0925 05:15:37.632083 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.632084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.632094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.632149 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.632772 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.633254 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.634328 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.634515 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.634581 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.637090 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:37.637092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:37.637092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:37.637098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:37.638464 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.638955 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.638957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.638958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.639016 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:37.639227 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:37.639227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:37.639227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:37.639334 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:37.639384 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.639584 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.639699 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":41}
I20260925 05:15:37.639812 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.631765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35197 (local address 127.0.0.1:50676)
0925 05:15:37.631927 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.631938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.632015 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.632178 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.632179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.632624 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:37.632688 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.632691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.633333 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.633334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.634215 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.634218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.634597 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.634598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.634670 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.635326 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:37.635331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:37.636989 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:37.638544 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.638553 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.638556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.638831 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.639082 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:37.639083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:37.639083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:37.639166 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:37.639393 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:37.639401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:37.639456 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.639530 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.639654 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260925 05:15:37.642500 21460 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:50676:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 05:15:37.642746 21460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:37.644377 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 507e03d7afff4a3b9a0a6f42636b65c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03e07baf64ca46a69246beb43df7a07f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:37.644376 21598 tablet_service.cc:1467] Processing CreateTablet for tablet 507e03d7afff4a3b9a0a6f42636b65c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03e07baf64ca46a69246beb43df7a07f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:37.644554 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507e03d7afff4a3b9a0a6f42636b65c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.644554 21598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507e03d7afff4a3b9a0a6f42636b65c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.645871 21973 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: Bootstrap starting.
I20260925 05:15:37.646132 21878 tablet_service.cc:1467] Processing CreateTablet for tablet 507e03d7afff4a3b9a0a6f42636b65c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03e07baf64ca46a69246beb43df7a07f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 05:15:37.646281 21878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507e03d7afff4a3b9a0a6f42636b65c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:37.646327 21973 tablet_bootstrap.cc:654] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.647013 21973 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: No bootstrap required, opened a new log
I20260925 05:15:37.647078 21973 ts_tablet_manager.cc:1397] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:37.647234 21973 raft_consensus.cc:348] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.647318 21973 raft_consensus.cc:374] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.647346 21973 raft_consensus.cc:729] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.647389 21973 consensus_queue.cc:260] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.647485 21973 ts_tablet_manager.cc:1428] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:37.647620 21976 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.647696 21976 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.647791 21972 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: Bootstrap starting.
I20260925 05:15:37.648200 21972 tablet_bootstrap.cc:654] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.648527 21978 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: Bootstrap starting.
I20260925 05:15:37.648864 21972 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: No bootstrap required, opened a new log
I20260925 05:15:37.648918 21978 tablet_bootstrap.cc:654] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:37.648931 21972 ts_tablet_manager.cc:1397] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:37.649080 21972 raft_consensus.cc:348] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.649163 21972 raft_consensus.cc:374] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.649191 21972 raft_consensus.cc:729] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.649232 21972 consensus_queue.cc:260] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.649325 21972 ts_tablet_manager.cc:1428] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:37.649430 21977 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.649510 21977 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.649752 21978 tablet_bootstrap.cc:492] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: No bootstrap required, opened a new log
I20260925 05:15:37.649791 21978 ts_tablet_manager.cc:1397] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:37.649931 21978 raft_consensus.cc:348] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.650002 21978 raft_consensus.cc:374] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:37.650020 21978 raft_consensus.cc:729] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713be8f34819496a8c0c4ae262c6cce8, State: Initialized, Role: FOLLOWER
I20260925 05:15:37.650056 21978 consensus_queue.cc:260] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.650174 21978 ts_tablet_manager.cc:1428] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:37.650225 21980 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.650321 21980 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 05:15:37.841300 21945 tablet.cc:2369] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:15:37.905946 21976 raft_consensus.cc:482] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:37.906075 21976 raft_consensus.cc:504] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.906452 21976 leader_election.cc:288] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:37.909621 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.907034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:49113 (local address 127.13.160.131:46875)
0925 05:15:37.907097 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.907099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.907107 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:37.907166 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.907166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.907175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.907246 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.907440 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.907917 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.908918 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.909274 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.909328 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.909335 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.909498 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.909543 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:37.909648 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.906784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:43935 (local address 127.13.160.129:45969)
0925 05:15:37.906858 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.906860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.907013 (+ 153us) server_negotiation.cc:407] Connection header received
0925 05:15:37.907151 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.907152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.907176 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.907253 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:37.907418 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.907898 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.908919 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.909287 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.909376 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.909385 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.909545 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.909588 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:15:37.909650 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.906958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:46875 (local address 127.13.160.130:49113)
0925 05:15:37.907048 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.907058 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.907108 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.907262 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.907263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.907265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.907345 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.907349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.908023 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.908024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.908819 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.908822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.909352 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.909353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.909425 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.909429 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.909485 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.909547 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":42,"thread_start_us":39,"threads_started":1}
I20260925 05:15:37.909659 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.906722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:45969 (local address 127.13.160.130:43935)
0925 05:15:37.906879 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.906938 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.907027 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.907259 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.907260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.907262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:37.907347 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.907349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.907984 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.907985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.908817 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.908819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.909391 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.909397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.909449 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.909452 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.909527 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.909565 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260925 05:15:37.909927 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507e03d7afff4a3b9a0a6f42636b65c2" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8" is_pre_election: true
I20260925 05:15:37.910029 21898 raft_consensus.cc:2454] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 0.
I20260925 05:15:37.910104 21618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507e03d7afff4a3b9a0a6f42636b65c2" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bffbe9e621a4bfd855cf2265281f159" is_pre_election: true
I20260925 05:15:37.910202 21692 leader_election.cc:302] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 713be8f34819496a8c0c4ae262c6cce8; no voters:
I20260925 05:15:37.910265 21618 raft_consensus.cc:2454] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 0.
I20260925 05:15:37.910339 21976 raft_consensus.cc:2781] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:37.910414 21976 raft_consensus.cc:482] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:37.910437 21976 raft_consensus.cc:3037] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.911288 21976 raft_consensus.cc:504] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.911420 21976 leader_election.cc:288] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 election: Requested vote from peers 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:37.911598 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507e03d7afff4a3b9a0a6f42636b65c2" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8"
I20260925 05:15:37.911599 21618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507e03d7afff4a3b9a0a6f42636b65c2" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bffbe9e621a4bfd855cf2265281f159"
I20260925 05:15:37.911680 21898 raft_consensus.cc:3037] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.911681 21618 raft_consensus.cc:3037] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:37.912454 21898 raft_consensus.cc:2454] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 1.
I20260925 05:15:37.912497 21618 raft_consensus.cc:2454] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 1.
I20260925 05:15:37.912627 21692 leader_election.cc:302] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:37.912734 21976 raft_consensus.cc:2781] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:37.912861 21976 raft_consensus.cc:686] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [term 1 LEADER]: Becoming Leader. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Running, Role: LEADER
I20260925 05:15:37.912968 21976 consensus_queue.cc:237] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:37.913304 22013 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: TxnStatusTablet state changed. Reason: New leader 360d959122294219a6abbb5fac2d88fb. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.913420 22013 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:37.913625 22015 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 05:15:37.913725 22015 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 05:15:37.913714 21461 catalog_manager.cc:5495] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 360d959122294219a6abbb5fac2d88fb (127.13.160.130). New cstate: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:37.937805 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.935039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.13.160.130:40903)
0925 05:15:37.935081 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.935083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.935126 (+ 43us) server_negotiation.cc:407] Connection header received
0925 05:15:37.935236 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.935236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.935253 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.935312 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:37.935473 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.935938 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.937060 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.937242 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.937290 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.937497 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.937596 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.937649 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.937726 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.937743 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:15:37.937860 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.934992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:40903 (local address 127.0.0.1:43098)
0925 05:15:37.935080 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.935092 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.935160 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.935342 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.935343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.935344 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:37.935401 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.935404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.936015 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.936016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.936973 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.936976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.937312 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.937313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.937386 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.937404 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.937658 (+ 254us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.937665 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.937722 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.937772 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 05:15:37.938962 21898 raft_consensus.cc:1261] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Refusing update from remote peer 360d959122294219a6abbb5fac2d88fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.939075 21618 raft_consensus.cc:1261] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Refusing update from remote peer 360d959122294219a6abbb5fac2d88fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.939237 21976 consensus_queue.cc:1035] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.939365 22013 consensus_queue.cc:1035] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.940088 21977 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: TxnStatusTablet state changed. Reason: New leader 360d959122294219a6abbb5fac2d88fb. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.940096 21980 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: TxnStatusTablet state changed. Reason: New leader 360d959122294219a6abbb5fac2d88fb. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.940197 21980 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.940198 21977 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.940649 22013 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.940732 22013 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:37.940970 21988 mvcc.cc:205] Tried to move back new op lower bound from 7333123432195764224 to 7333123432092053504. Current Snapshot: MvccSnapshot[applied={T|T < 7333123432195764224}]
I20260925 05:15:37.941076 21980 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.941139 21980 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 713be8f34819496a8c0c4ae262c6cce8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.942209 21976 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.942294 21976 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 360d959122294219a6abbb5fac2d88fb: This TxnStatusTablet replica's current role is: LEADER
I20260925 05:15:37.942992 21977 tablet_replica.cc:439] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } } }
I20260925 05:15:37.943096 21977 tablet_replica.cc:442] T 507e03d7afff4a3b9a0a6f42636b65c2 P 8bffbe9e621a4bfd855cf2265281f159: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 05:15:37.947010 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.944505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:50423 (local address 127.13.160.130:40903)
0925 05:15:37.944543 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.944545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.944704 (+ 159us) server_negotiation.cc:407] Connection header received
0925 05:15:37.944781 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.944782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.944793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.944847 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:37.944960 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.945468 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.946254 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.946461 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.946547 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.946734 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.946775 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.946838 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.946916 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.946940 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 05:15:37.947063 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.944465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:40903 (local address 127.13.160.129:50423)
0925 05:15:37.944630 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.944639 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.944721 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.944849 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.944850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.944852 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:37.944902 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.944904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.945556 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.945557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.946168 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.946170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.946549 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.946551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.946607 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.946634 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.946838 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.946840 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.946921 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.946971 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260925 05:15:37.949131 21898 raft_consensus.cc:1261] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.949143 21758 raft_consensus.cc:1261] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:37.949355 21983 consensus_queue.cc:1035] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.949460 21977 consensus_queue.cc:1035] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:37.957911 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.955130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:34007 (local address 127.13.160.129:45969)
0925 05:15:37.955174 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.955176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.955325 (+ 149us) server_negotiation.cc:407] Connection header received
0925 05:15:37.955404 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.955405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.955416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.955472 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:37.955660 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.956122 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.957255 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.957437 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.957491 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.957695 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.957722 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.957759 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:37.957835 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 05:15:37.957853 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:15:37.958072 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.955094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:45969 (local address 127.13.160.130:34007)
0925 05:15:37.955247 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:37.955258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:37.955340 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:37.955521 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:37.955522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:37.955524 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 05:15:37.955588 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.955590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.956203 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.956204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.957172 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:37.957175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.957522 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.957523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:37.957593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.957612 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 05:15:37.957778 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 05:15:37.957780 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:37.957834 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 05:15:37.957879 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260925 05:15:37.966320 21460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:37.966450 21460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:37.967116 21460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Sending DeleteTablet for 3 replicas of tablet 96d10876dbc94529b09f1957b0bfbda5
I20260925 05:15:37.967447 21598 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:40478
I20260925 05:15:37.967552 21997 tablet_replica.cc:330] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: stopping tablet replica
I20260925 05:15:37.967598 21997 raft_consensus.cc:2229] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:37.967661 21997 raft_consensus.cc:2258] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.967779 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:43084
I20260925 05:15:37.967865 21996 tablet_replica.cc:330] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: stopping tablet replica
I20260925 05:15:37.967907 21996 raft_consensus.cc:2229] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:37.967945 21996 raft_consensus.cc:2258] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.968246 21996 ts_tablet_manager.cc:1905] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.970110 21997 ts_tablet_manager.cc:1905] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.971698 21997 ts_tablet_manager.cc:1918] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:37.971766 21997 log.cc:1196] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/96d10876dbc94529b09f1957b0bfbda5
I20260925 05:15:37.971980 21997 ts_tablet_manager.cc:1939] T 96d10876dbc94529b09f1957b0bfbda5 P 8bffbe9e621a4bfd855cf2265281f159: Deleting consensus metadata
I20260925 05:15:37.972332 21448 catalog_manager.cc:4841] TS 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969): tablet 96d10876dbc94529b09f1957b0bfbda5 (table test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]) successfully deleted
I20260925 05:15:37.972373 21878 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d10876dbc94529b09f1957b0bfbda5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:37 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:35704
I20260925 05:15:37.972416 21996 ts_tablet_manager.cc:1918] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:37.972462 21995 tablet_replica.cc:330] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: stopping tablet replica
I20260925 05:15:37.972483 21996 log.cc:1196] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/96d10876dbc94529b09f1957b0bfbda5
I20260925 05:15:37.972504 21995 raft_consensus.cc:2229] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:37.972548 21995 raft_consensus.cc:2258] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:37.972693 21996 ts_tablet_manager.cc:1939] T 96d10876dbc94529b09f1957b0bfbda5 P 360d959122294219a6abbb5fac2d88fb: Deleting consensus metadata
I20260925 05:15:37.972901 21995 ts_tablet_manager.cc:1905] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:37.973008 21447 catalog_manager.cc:4841] TS 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903): tablet 96d10876dbc94529b09f1957b0bfbda5 (table test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]) successfully deleted
I20260925 05:15:37.974231 21995 ts_tablet_manager.cc:1918] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:37.974301 21995 log.cc:1196] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/96d10876dbc94529b09f1957b0bfbda5
I20260925 05:15:37.974515 21995 ts_tablet_manager.cc:1939] T 96d10876dbc94529b09f1957b0bfbda5 P 713be8f34819496a8c0c4ae262c6cce8: Deleting consensus metadata
I20260925 05:15:37.974865 21448 catalog_manager.cc:4841] TS 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875): tablet 96d10876dbc94529b09f1957b0bfbda5 (table test-table [id=7c45b18c57ba455e9304c7bc1514a5ac]) successfully deleted
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 05:15:37.995160 21515 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:37.989285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.13.160.190:35197)
0925 05:15:37.989353 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.989355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.989455 (+ 100us) server_negotiation.cc:407] Connection header received
0925 05:15:37.989494 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.989495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.989507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.989586 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.990295 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.991149 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.991888 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.992181 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.992259 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:37.995078 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50680: BlockingRecv error: recv got EOF from 127.0.0.1:50680 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 05:15:37 dist-test-slave-9tgz krb5kdc[21416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM
I20260925 05:15:38.003638 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:37.996046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50688 (local address 127.13.160.190:35197)
0925 05:15:37.996088 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:37.996090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:37.996171 (+ 81us) server_negotiation.cc:407] Connection header received
0925 05:15:37.996230 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:37.996231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:37.996241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:37.996300 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:37.996732 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.997497 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.998192 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:37.998449 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:37.998525 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:38.000933 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:38.000935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:38.000935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:38.000943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:38.002499 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:38.002835 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:38.002837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:38.002837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:38.002931 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:38.003123 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:38.003124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:38.003124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:38.003277 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:38.003394 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:38.003460 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 05:15:38.003538 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:15:38.005250 21460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:38.005451 21460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:38.006814 21738 tablet_service.cc:1467] Processing CreateTablet for tablet d4fa81f61f874349b1b3655deadf3f0b (DEFAULT_TABLE table=test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.006840 21598 tablet_service.cc:1467] Processing CreateTablet for tablet d4fa81f61f874349b1b3655deadf3f0b (DEFAULT_TABLE table=test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.006989 21598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fa81f61f874349b1b3655deadf3f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.006990 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fa81f61f874349b1b3655deadf3f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.006973 21878 tablet_service.cc:1467] Processing CreateTablet for tablet d4fa81f61f874349b1b3655deadf3f0b (DEFAULT_TABLE table=test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.007107 21878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4fa81f61f874349b1b3655deadf3f0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.008308 21972 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Bootstrap starting.
I20260925 05:15:38.008319 21973 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Bootstrap starting.
I20260925 05:15:38.008546 21978 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Bootstrap starting.
I20260925 05:15:38.008677 21972 tablet_bootstrap.cc:654] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.008781 21973 tablet_bootstrap.cc:654] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.008939 21978 tablet_bootstrap.cc:654] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.009766 21973 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: No bootstrap required, opened a new log
I20260925 05:15:38.009824 21973 ts_tablet_manager.cc:1397] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:38.010011 21973 raft_consensus.cc:348] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.010092 21973 raft_consensus.cc:374] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.010119 21973 raft_consensus.cc:729] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.010166 21973 consensus_queue.cc:260] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.010257 21973 ts_tablet_manager.cc:1428] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:38.010603 21978 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: No bootstrap required, opened a new log
I20260925 05:15:38.010673 21978 ts_tablet_manager.cc:1397] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:38.010828 21978 raft_consensus.cc:348] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.010963 21978 raft_consensus.cc:374] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.010991 21978 raft_consensus.cc:729] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713be8f34819496a8c0c4ae262c6cce8, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.011042 21978 consensus_queue.cc:260] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.011148 21978 ts_tablet_manager.cc:1428] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:38.015401 21972 tablet_bootstrap.cc:492] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: No bootstrap required, opened a new log
I20260925 05:15:38.015460 21972 ts_tablet_manager.cc:1397] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260925 05:15:38.015609 21972 raft_consensus.cc:348] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.015722 21972 raft_consensus.cc:374] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.015753 21972 raft_consensus.cc:729] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.015811 21972 consensus_queue.cc:260] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.015909 21972 ts_tablet_manager.cc:1428] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:38.159294 22013 raft_consensus.cc:482] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:38.159421 22013 raft_consensus.cc:504] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.159595 22013 leader_election.cc:288] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:38.159801 21618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fa81f61f874349b1b3655deadf3f0b" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bffbe9e621a4bfd855cf2265281f159" is_pre_election: true
I20260925 05:15:38.159842 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fa81f61f874349b1b3655deadf3f0b" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8" is_pre_election: true
I20260925 05:15:38.159914 21898 raft_consensus.cc:2454] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 0.
I20260925 05:15:38.159925 21618 raft_consensus.cc:2454] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 0.
I20260925 05:15:38.160084 21692 leader_election.cc:302] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:38.160230 22013 raft_consensus.cc:2781] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:38.160282 22013 raft_consensus.cc:482] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:38.160301 22013 raft_consensus.cc:3037] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.161027 22013 raft_consensus.cc:504] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.161175 22013 leader_election.cc:288] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 election: Requested vote from peers 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:38.161358 21618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fa81f61f874349b1b3655deadf3f0b" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bffbe9e621a4bfd855cf2265281f159"
I20260925 05:15:38.161393 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4fa81f61f874349b1b3655deadf3f0b" candidate_uuid: "360d959122294219a6abbb5fac2d88fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8"
I20260925 05:15:38.161463 21618 raft_consensus.cc:3037] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.161463 21898 raft_consensus.cc:3037] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.162045 21898 raft_consensus.cc:2454] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 1.
I20260925 05:15:38.162213 21618 raft_consensus.cc:2454] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360d959122294219a6abbb5fac2d88fb in term 1.
I20260925 05:15:38.162232 21692 leader_election.cc:302] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 713be8f34819496a8c0c4ae262c6cce8; no voters:
I20260925 05:15:38.162371 22013 raft_consensus.cc:2781] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:38.162431 22013 raft_consensus.cc:686] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 1 LEADER]: Becoming Leader. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Running, Role: LEADER
I20260925 05:15:38.162480 22013 consensus_queue.cc:237] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.163091 21460 catalog_manager.cc:5495] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 360d959122294219a6abbb5fac2d88fb (127.13.160.130). New cstate: current_term: 1 leader_uuid: "360d959122294219a6abbb5fac2d88fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:38.199365 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:38.197095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.13.160.130:40903)
0925 05:15:38.197172 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:38.197175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:38.197184 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:38.197217 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:38.197218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:38.197229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:38.197291 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:38.197407 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:38.197861 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:38.198619 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:38.198810 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:38.198927 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:38.199078 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:38.199115 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:38.199189 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:38.199252 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 05:15:38.199273 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:15:38.200289 21898 raft_consensus.cc:1261] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Refusing update from remote peer 360d959122294219a6abbb5fac2d88fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:38.200322 21618 raft_consensus.cc:1261] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Refusing update from remote peer 360d959122294219a6abbb5fac2d88fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:38.200560 22013 consensus_queue.cc:1035] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:38.200702 22023 consensus_queue.cc:1035] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:38.204170 21460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:38.204289 21460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:38.204972 21460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Sending DeleteTablet for 3 replicas of tablet d4fa81f61f874349b1b3655deadf3f0b
I20260925 05:15:38.205221 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fa81f61f874349b1b3655deadf3f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:43084
I20260925 05:15:38.205350 21996 tablet_replica.cc:330] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: stopping tablet replica
I20260925 05:15:38.205345 21878 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fa81f61f874349b1b3655deadf3f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:35704
I20260925 05:15:38.205353 21598 tablet_service.cc:1514] Processing DeleteTablet for tablet d4fa81f61f874349b1b3655deadf3f0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:40478
I20260925 05:15:38.205412 21996 raft_consensus.cc:2229] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:38.205456 21997 tablet_replica.cc:330] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: stopping tablet replica
I20260925 05:15:38.205459 21995 tablet_replica.cc:330] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: stopping tablet replica
I20260925 05:15:38.205507 21997 raft_consensus.cc:2229] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:38.205515 21996 raft_consensus.cc:2258] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.205516 21995 raft_consensus.cc:2229] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:38.205566 21995 raft_consensus.cc:2258] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.205556 21997 raft_consensus.cc:2258] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.205870 21995 ts_tablet_manager.cc:1905] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.205870 21996 ts_tablet_manager.cc:1905] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.206058 21997 ts_tablet_manager.cc:1905] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.207597 21996 ts_tablet_manager.cc:1918] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:38.207599 21995 ts_tablet_manager.cc:1918] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:38.207664 21996 log.cc:1196] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/d4fa81f61f874349b1b3655deadf3f0b
I20260925 05:15:38.207674 21995 log.cc:1196] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/d4fa81f61f874349b1b3655deadf3f0b
I20260925 05:15:38.207721 21460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:38.207893 21460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:38.207909 21996 ts_tablet_manager.cc:1939] T d4fa81f61f874349b1b3655deadf3f0b P 360d959122294219a6abbb5fac2d88fb: Deleting consensus metadata
I20260925 05:15:38.207923 21995 ts_tablet_manager.cc:1939] T d4fa81f61f874349b1b3655deadf3f0b P 713be8f34819496a8c0c4ae262c6cce8: Deleting consensus metadata
I20260925 05:15:38.208211 21997 ts_tablet_manager.cc:1918] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:15:38.208244 21447 catalog_manager.cc:4841] TS 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903): tablet d4fa81f61f874349b1b3655deadf3f0b (table test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]) successfully deleted
I20260925 05:15:38.208273 21997 log.cc:1196] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/d4fa81f61f874349b1b3655deadf3f0b
I20260925 05:15:38.208297 21448 catalog_manager.cc:4841] TS 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875): tablet d4fa81f61f874349b1b3655deadf3f0b (table test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]) successfully deleted
I20260925 05:15:38.208482 21997 ts_tablet_manager.cc:1939] T d4fa81f61f874349b1b3655deadf3f0b P 8bffbe9e621a4bfd855cf2265281f159: Deleting consensus metadata
I20260925 05:15:38.208848 21448 catalog_manager.cc:4841] TS 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969): tablet d4fa81f61f874349b1b3655deadf3f0b (table test-table [id=068309cfcf6a4b8e92d9b494aa168f8a]) successfully deleted
I20260925 05:15:38.209404 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1f619e0023476d820931844eac1201 (DEFAULT_TABLE table=test-table [id=a321c57027f849eca4d7bd0835364d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.209416 21598 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1f619e0023476d820931844eac1201 (DEFAULT_TABLE table=test-table [id=a321c57027f849eca4d7bd0835364d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.209517 21598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1f619e0023476d820931844eac1201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.209538 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1f619e0023476d820931844eac1201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.210566 21973 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Bootstrap starting.
I20260925 05:15:38.210844 21878 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1f619e0023476d820931844eac1201 (DEFAULT_TABLE table=test-table [id=a321c57027f849eca4d7bd0835364d8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:38.210996 21878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1f619e0023476d820931844eac1201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:38.211030 21973 tablet_bootstrap.cc:654] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.211315 21972 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Bootstrap starting.
I20260925 05:15:38.211691 21972 tablet_bootstrap.cc:654] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.212072 21973 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: No bootstrap required, opened a new log
I20260925 05:15:38.212126 21973 ts_tablet_manager.cc:1397] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 05:15:38.212249 21978 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Bootstrap starting.
I20260925 05:15:38.212251 21973 raft_consensus.cc:348] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.212318 21973 raft_consensus.cc:374] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.212337 21973 raft_consensus.cc:729] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360d959122294219a6abbb5fac2d88fb, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.212379 21973 consensus_queue.cc:260] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.212486 21973 ts_tablet_manager.cc:1428] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:38.212728 21972 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: No bootstrap required, opened a new log
I20260925 05:15:38.212772 21972 ts_tablet_manager.cc:1397] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:38.212873 21972 raft_consensus.cc:348] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.212947 21972 raft_consensus.cc:374] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.212946 21978 tablet_bootstrap.cc:654] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:38.212970 21972 raft_consensus.cc:729] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.213022 21972 consensus_queue.cc:260] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.213135 21972 ts_tablet_manager.cc:1428] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:38.213747 21978 tablet_bootstrap.cc:492] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: No bootstrap required, opened a new log
I20260925 05:15:38.213814 21978 ts_tablet_manager.cc:1397] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:38.213950 21978 raft_consensus.cc:348] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.214039 21978 raft_consensus.cc:374] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:38.214061 21978 raft_consensus.cc:729] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713be8f34819496a8c0c4ae262c6cce8, State: Initialized, Role: FOLLOWER
I20260925 05:15:38.214112 21978 consensus_queue.cc:260] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.214205 21978 ts_tablet_manager.cc:1428] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:38.232311 21983 raft_consensus.cc:482] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:38.232432 21983 raft_consensus.cc:504] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.232662 21983 leader_election.cc:288] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:38.232812 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1f619e0023476d820931844eac1201" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb" is_pre_election: true
I20260925 05:15:38.232852 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1f619e0023476d820931844eac1201" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8" is_pre_election: true
I20260925 05:15:38.232928 21758 raft_consensus.cc:2454] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:38.232947 21898 raft_consensus.cc:2454] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 0.
I20260925 05:15:38.233069 21551 leader_election.cc:302] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360d959122294219a6abbb5fac2d88fb, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:38.233183 21983 raft_consensus.cc:2781] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:38.233230 21983 raft_consensus.cc:482] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:38.233273 21983 raft_consensus.cc:3037] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.233994 21983 raft_consensus.cc:504] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.234148 21983 leader_election.cc:288] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Requested vote from peers 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903), 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875)
I20260925 05:15:38.234293 21758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1f619e0023476d820931844eac1201" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360d959122294219a6abbb5fac2d88fb"
I20260925 05:15:38.234380 21758 raft_consensus.cc:3037] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.234347 21898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1f619e0023476d820931844eac1201" candidate_uuid: "8bffbe9e621a4bfd855cf2265281f159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713be8f34819496a8c0c4ae262c6cce8"
I20260925 05:15:38.234426 21898 raft_consensus.cc:3037] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:38.235111 21898 raft_consensus.cc:2454] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:38.235180 21758 raft_consensus.cc:2454] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bffbe9e621a4bfd855cf2265281f159 in term 1.
I20260925 05:15:38.235268 21552 leader_election.cc:302] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713be8f34819496a8c0c4ae262c6cce8, 8bffbe9e621a4bfd855cf2265281f159; no voters:
I20260925 05:15:38.235387 21983 raft_consensus.cc:2781] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:38.235440 21983 raft_consensus.cc:686] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Becoming Leader. State: Replica: 8bffbe9e621a4bfd855cf2265281f159, State: Running, Role: LEADER
I20260925 05:15:38.235493 21983 consensus_queue.cc:237] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } }
I20260925 05:15:38.236126 21461 catalog_manager.cc:5495] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "8bffbe9e621a4bfd855cf2265281f159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bffbe9e621a4bfd855cf2265281f159" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 45969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260925 05:15:38.247757 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:38.245179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40492 (local address 127.13.160.129:45969)
0925 05:15:38.245224 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:38.245226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:38.245235 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:38.245259 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:38.245260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:38.245284 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:38.245349 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:15:38.245441 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:38.246193 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:38.247008 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:38.247323 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:38.247394 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:38.247516 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:15:38.247551 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:15:38.247599 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:38.247651 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 05:15:38.247673 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:38.249814 21898 raft_consensus.cc:1261] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:38.249953 21758 raft_consensus.cc:1261] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Refusing update from remote peer 8bffbe9e621a4bfd855cf2265281f159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:15:38.250039 21977 consensus_queue.cc:1035] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "713be8f34819496a8c0c4ae262c6cce8" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:38.250172 22028 consensus_queue.cc:1035] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360d959122294219a6abbb5fac2d88fb" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:15:38.260656 21461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:38.260782 21461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:15:38.261402 21461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006faed066344638a413550eeeefd8a2: Sending DeleteTablet for 3 replicas of tablet 8b1f619e0023476d820931844eac1201
I20260925 05:15:38.261683 21598 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b1f619e0023476d820931844eac1201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:40478
I20260925 05:15:38.261811 21997 tablet_replica.cc:330] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: stopping tablet replica
I20260925 05:15:38.261891 21997 raft_consensus.cc:2229] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:15:38.261960 21997 raft_consensus.cc:2258] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.261974 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b1f619e0023476d820931844eac1201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:43084
I20260925 05:15:38.262107 21996 tablet_replica.cc:330] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: stopping tablet replica
I20260925 05:15:38.262121 21878 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b1f619e0023476d820931844eac1201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:15:38 UTC) from {username='oryx', principal='oryx/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:35704
I20260925 05:15:38.262178 21996 raft_consensus.cc:2229] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:38.262219 21996 raft_consensus.cc:2258] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.262219 21995 tablet_replica.cc:330] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: stopping tablet replica
I20260925 05:15:38.262261 21995 raft_consensus.cc:2229] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:15:38.262300 21995 raft_consensus.cc:2258] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:15:38.262662 21996 ts_tablet_manager.cc:1905] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.263092 21997 ts_tablet_manager.cc:1905] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.264166 21996 ts_tablet_manager.cc:1918] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:38.264264 21996 log.cc:1196] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/8b1f619e0023476d820931844eac1201
I20260925 05:15:38.264472 21996 ts_tablet_manager.cc:1939] T 8b1f619e0023476d820931844eac1201 P 360d959122294219a6abbb5fac2d88fb: Deleting consensus metadata
I20260925 05:15:38.264504 21997 ts_tablet_manager.cc:1918] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:38.264564 21997 log.cc:1196] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/8b1f619e0023476d820931844eac1201
I20260925 05:15:38.264781 21997 ts_tablet_manager.cc:1939] T 8b1f619e0023476d820931844eac1201 P 8bffbe9e621a4bfd855cf2265281f159: Deleting consensus metadata
I20260925 05:15:38.264830 21447 catalog_manager.cc:4841] TS 360d959122294219a6abbb5fac2d88fb (127.13.160.130:40903): tablet 8b1f619e0023476d820931844eac1201 (table test-table [id=a321c57027f849eca4d7bd0835364d8e]) successfully deleted
I20260925 05:15:38.264976 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21533
I20260925 05:15:38.271226 21995 ts_tablet_manager.cc:1905] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:15:38.272552 21995 ts_tablet_manager.cc:1918] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 05:15:38.272665 21995 log.cc:1196] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/8b1f619e0023476d820931844eac1201
I20260925 05:15:38.272888 21995 ts_tablet_manager.cc:1939] T 8b1f619e0023476d820931844eac1201 P 713be8f34819496a8c0c4ae262c6cce8: Deleting consensus metadata
W20260925 05:15:38.273229 21448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:45969 (error 108)
I20260925 05:15:38.273414 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21673
I20260925 05:15:38.273480 21448 catalog_manager.cc:4841] TS 713be8f34819496a8c0c4ae262c6cce8 (127.13.160.131:46875): tablet 8b1f619e0023476d820931844eac1201 (table test-table [id=a321c57027f849eca4d7bd0835364d8e]) successfully deleted
I20260925 05:15:38.273875 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:38.273641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:45969 (local address 127.0.0.1:40494)
0925 05:15:38.273793 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:38.273831 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":39,"threads_started":1}
W20260925 05:15:38.273952 21448 catalog_manager.cc:4584] TS 8bffbe9e621a4bfd855cf2265281f159 (127.13.160.129:45969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b1f619e0023476d820931844eac1201: Network error: Client connection negotiation failed: client connection to 127.13.160.129:45969: connect: Connection refused (error 111)
I20260925 05:15:38.279325 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21813
I20260925 05:15:38.284376 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 21429
2026-09-25T05:15:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2767 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:38 dist-test-slave-9tgz krb5kdc[22060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:38 dist-test-slave-9tgz krb5kdc[22060](info): set up 2 sockets
Sep 25 05:15:38 dist-test-slave-9tgz krb5kdc[22060](info): commencing operation
krb5kdc: starting...
W20260925 05:15:39.320744 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:39 dist-test-slave-9tgz krb5kdc[22060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:39.444278 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43827
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:43827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:39.529678 22073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:39.529919 22073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:39.529946 22073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:39.531246 22073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:39.531306 22073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:39.531322 22073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:39.531337 22073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:39.531384 22073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:39.533020 22073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:43827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:39.533331 22073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:39.533613 22073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:39.539000 22079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:39.539029 22081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:39.539052 22073 server_base.cc:1034] running on GCE node
W20260925 05:15:39.539000 22078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:39.539512 22073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:39.539848 22073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:39.541009 22073 hybrid_clock.cc:648] HybridClock initialized: now 1790313339540995 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T05:15:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1255 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:39 dist-test-slave-9tgz krb5kdc[22086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:39 dist-test-slave-9tgz krb5kdc[22086](info): set up 2 sockets
Sep 25 05:15:39 dist-test-slave-9tgz krb5kdc[22086](info): commencing operation
krb5kdc: starting...
W20260925 05:15:40.592665 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:40 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:40.714035 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45441
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:45441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:40.797861 22099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:40.798103 22099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:40.798130 22099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:40.799428 22099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:40.799480 22099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:40.799497 22099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:40.799511 22099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:40.799525 22099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:40.801012 22099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:45441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:40.801236 22099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:40.801422 22099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:40.806459 22105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:40.806447 22107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:40.806442 22104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:40.806682 22099 server_base.cc:1034] running on GCE node
I20260925 05:15:40.806963 22099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:40.807282 22099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:40.808427 22099 hybrid_clock.cc:648] HybridClock initialized: now 1790313340808416 us; error 32 us; skew 500 ppm
Sep 25 05:15:40 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:40.810881 22099 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:40.811378 22099 webserver.cc:460] Webserver started at http://127.13.160.190:41613/ using document root <none> and password file <none>
I20260925 05:15:40.811628 22099 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:40.811700 22099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:40.811846 22099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:40.812950 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "e342f5d1456941ca9f5d65bf70f4347d"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "492d6a07aefdd3a54ec6e1a857bb11ee"
server_key_iv: "f2a1b09029fbb54198efd7a212bebfff"
server_key_version: "encryptionkey@0"
I20260925 05:15:40.813335 22099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "e342f5d1456941ca9f5d65bf70f4347d"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "492d6a07aefdd3a54ec6e1a857bb11ee"
server_key_iv: "f2a1b09029fbb54198efd7a212bebfff"
server_key_version: "encryptionkey@0"
I20260925 05:15:40.814692 22099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:40.815514 22114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:40.815699 22099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:40.815795 22099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "e342f5d1456941ca9f5d65bf70f4347d"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "492d6a07aefdd3a54ec6e1a857bb11ee"
server_key_iv: "f2a1b09029fbb54198efd7a212bebfff"
server_key_version: "encryptionkey@0"
I20260925 05:15:40.815879 22099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:40.825975 22099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:40.827987 22099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:40.828173 22099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:40.832470 22099 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:45441
I20260925 05:15:40.832496 22166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:45441 every 8 connection(s)
I20260925 05:15:40.832875 22099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:40.833788 22167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:40.836243 22167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Bootstrap starting.
I20260925 05:15:40.836844 22167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:40.837141 22167 log.cc:824] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:40.837865 22167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: No bootstrap required, opened a new log
I20260925 05:15:40.838929 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22099
I20260925 05:15:40.839046 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:40.839221 13954 external_mini_cluster.cc:1428] Setting key 6307402d84d7f98f64eccb827d913bc4
I20260925 05:15:40.839999 22167 raft_consensus.cc:348] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:40.840186 22167 raft_consensus.cc:374] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:40.840221 22167 raft_consensus.cc:729] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e342f5d1456941ca9f5d65bf70f4347d, State: Initialized, Role: FOLLOWER
I20260925 05:15:40.840353 22167 consensus_queue.cc:260] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:40.840440 22167 raft_consensus.cc:388] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:40.840472 22167 raft_consensus.cc:482] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:40.840528 22167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:40.841581 22167 raft_consensus.cc:504] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:40.841693 22167 leader_election.cc:302] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e342f5d1456941ca9f5d65bf70f4347d; no voters:
I20260925 05:15:40.841845 22167 leader_election.cc:288] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:40.841909 22172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:40.842044 22172 raft_consensus.cc:686] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 LEADER]: Becoming Leader. State: Replica: e342f5d1456941ca9f5d65bf70f4347d, State: Running, Role: LEADER
I20260925 05:15:40.842082 22167 sys_catalog.cc:564] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:40.842141 22172 consensus_queue.cc:237] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:40.842466 22172 sys_catalog.cc:455] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e342f5d1456941ca9f5d65bf70f4347d. Latest consensus state: current_term: 1 leader_uuid: "e342f5d1456941ca9f5d65bf70f4347d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } } }
I20260925 05:15:40.842556 22172 sys_catalog.cc:458] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:40.842478 22174 sys_catalog.cc:455] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e342f5d1456941ca9f5d65bf70f4347d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } } }
I20260925 05:15:40.842710 22174 sys_catalog.cc:458] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:40.842811 22179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:40.843375 22179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:40.845055 22179 catalog_manager.cc:1347] Generated new cluster ID: 434dc103b66646d1a8301ad03445d0e5
I20260925 05:15:40.845105 22179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:40 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:40.851233 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:40.839757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.13.160.190:45441)
0925 05:15:40.839971 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:40.839981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:40.840015 (+ 34us) server_negotiation.cc:407] Connection header received
0925 05:15:40.840131 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:40.840139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:40.840196 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:40.840347 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:40.841027 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:40.842422 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:40.843688 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:40.844266 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:40.844378 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:40.847245 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:40.847249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:40.847250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:40.847264 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:40.849349 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:40.849759 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:40.849763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:40.849763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:40.849919 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:40.850121 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:40.850122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:40.850122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:40.850459 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:40.850633 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:40.850738 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 05:15:40.850954 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260925 05:15:40.858692 22179 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:40.859467 22179 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:40.869309 22179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Generated new TSK 0
I20260925 05:15:40.869555 22179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:40.899732 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:40.982744 22193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:40.982996 22193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:40.983024 22193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:40.984258 22193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:40.984302 22193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:40.984331 22193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:40.985855 22193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:40.986121 22193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:40.986363 22193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:40.991396 22199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:40.991387 22201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:40.991387 22198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:40.991854 22193 server_base.cc:1034] running on GCE node
I20260925 05:15:40.992108 22193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:40.992357 22193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:40.993530 22193 hybrid_clock.cc:648] HybridClock initialized: now 1790313340993515 us; error 30 us; skew 500 ppm
Sep 25 05:15:40 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:40.996250 22193 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:40.996838 22193 webserver.cc:460] Webserver started at http://127.13.160.129:37571/ using document root <none> and password file <none>
I20260925 05:15:40.997143 22193 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:40.997211 22193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:40.997361 22193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:40.998528 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "f65639be70fd4cb5badec2a0afccadab"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "7822436dbd6d1d430e5cb7b717860fff"
server_key_iv: "1ac9b29bed1995c3ea9d17215c99efaa"
server_key_version: "encryptionkey@0"
I20260925 05:15:40.998945 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "f65639be70fd4cb5badec2a0afccadab"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "7822436dbd6d1d430e5cb7b717860fff"
server_key_iv: "1ac9b29bed1995c3ea9d17215c99efaa"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.000298 22193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:41.001056 22208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.001230 22193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:41.001322 22193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "f65639be70fd4cb5badec2a0afccadab"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "7822436dbd6d1d430e5cb7b717860fff"
server_key_iv: "1ac9b29bed1995c3ea9d17215c99efaa"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.001402 22193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.014504 22193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.017057 22193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.017242 22193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.017500 22193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.017823 22193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.017856 22193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.017899 22193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.017926 22193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.023865 22193 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:38375
I20260925 05:15:41.023913 22321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:38375 every 8 connection(s)
I20260925 05:15:41.024251 22193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:41.025252 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22193
I20260925 05:15:41.025362 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:41.025555 13954 external_mini_cluster.cc:1428] Setting key 520869479747376924769d9d3dac25d5
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313340, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
I20260925 05:15:41.034807 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.025108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:60411 (local address 127.13.160.190:45441)
0925 05:15:41.025200 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.025203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.025491 (+ 288us) server_negotiation.cc:407] Connection header received
0925 05:15:41.026086 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.026088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.026102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.026169 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.027230 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.027730 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.028744 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.028964 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.029053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.031592 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:41.031594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:41.031595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:41.031605 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:41.033328 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.033838 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.033840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.033840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.033903 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.034182 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.034183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.034184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.034298 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:41.034354 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:41.034623 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 05:15:41.034707 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 05:15:41.035137 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.024976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:60411)
0925 05:15:41.025341 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.025392 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.025974 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.026247 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.026251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.026852 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.027130 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.027135 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.027840 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.027841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.028638 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.028641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.029065 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.029066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.029194 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.029762 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.029771 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.031485 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:41.033437 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.033446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.033448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.033706 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.033981 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.033982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.033982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.034107 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.034363 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:41.034367 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:41.034499 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 05:15:41.034598 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 05:15:41.034779 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":96,"threads_started":1}
I20260925 05:15:41.035781 22322 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45441
I20260925 05:15:41.035935 22322 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:41.036159 22322 heartbeater.cc:511] Master 127.13.160.190:45441 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
I20260925 05:15:41.036794 22131 ts_manager.cc:194] Registered new tserver with Master: f65639be70fd4cb5badec2a0afccadab (127.13.160.129:38375)
I20260925 05:15:41.037421 22131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:60411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:41.052006 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:41.133822 22327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.134050 22327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.134076 22327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.135320 22327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.135370 22327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.135397 22327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:41.136917 22327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.137181 22327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.137429 22327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:41.142581 22327 server_base.cc:1034] running on GCE node
W20260925 05:15:41.142561 22333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.142563 22332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.142561 22335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.143126 22327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.143360 22327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.144531 22327 hybrid_clock.cc:648] HybridClock initialized: now 1790313341144520 us; error 42 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.147152 22327 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:41.147603 22327 webserver.cc:460] Webserver started at http://127.13.160.130:44495/ using document root <none> and password file <none>
I20260925 05:15:41.147843 22327 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.147910 22327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.148072 22327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:41.149148 22327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "13fb0c0595c94c8aab07465e062c831e"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "a7d203957f40c35a639b134fa2b3b81e"
server_key_iv: "6633a218066fa4f9c58a7219701f5c7d"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.149521 22327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "13fb0c0595c94c8aab07465e062c831e"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "a7d203957f40c35a639b134fa2b3b81e"
server_key_iv: "6633a218066fa4f9c58a7219701f5c7d"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.150839 22327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:41.151590 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.151767 22327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:41.151867 22327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "13fb0c0595c94c8aab07465e062c831e"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "a7d203957f40c35a639b134fa2b3b81e"
server_key_iv: "6633a218066fa4f9c58a7219701f5c7d"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.151950 22327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.169909 22327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.171931 22327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.172104 22327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.172359 22327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.172689 22327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.172744 22327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.172780 22327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.172806 22327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.178460 22327 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:45027
I20260925 05:15:41.178504 22455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:45027 every 8 connection(s)
I20260925 05:15:41.178831 22327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:41.187378 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22327
I20260925 05:15:41.187489 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:41.187678 13954 external_mini_cluster.cc:1428] Setting key 8df829bf556ae97049b1396588999234
I20260925 05:15:41.187820 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.179627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:45185 (local address 127.13.160.190:45441)
0925 05:15:41.179688 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.179690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.179847 (+ 157us) server_negotiation.cc:407] Connection header received
0925 05:15:41.180340 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.180341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.180350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.180405 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.181318 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.181821 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.182765 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.182964 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.183037 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.185114 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:41.185115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:41.185115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:41.185121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:41.186332 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.186705 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.186706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.186706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.186756 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.187019 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.187020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.187020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.187124 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:41.187181 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:41.187613 (+ 432us) server_negotiation.cc:299] Negotiation successful
0925 05:15:41.187691 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:15:41.188035 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.179570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:45185)
0925 05:15:41.179735 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.179778 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.180255 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.180470 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.180473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.181038 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.181243 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.181247 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.181901 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.181902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.182693 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.182695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.183037 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.183038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.183136 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.183683 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.183699 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.185033 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:41.186402 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.186407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.186409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.186616 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.186814 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.186815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.186815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.186964 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.187291 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:41.187298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:41.187478 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 05:15:41.187600 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 05:15:41.187784 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":103,"threads_started":1}
I20260925 05:15:41.188531 22456 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45441
I20260925 05:15:41.188637 22456 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:41.188835 22456 heartbeater.cc:511] Master 127.13.160.190:45441 requested a full tablet report, sending...
I20260925 05:15:41.189236 22131 ts_manager.cc:194] Registered new tserver with Master: 13fb0c0595c94c8aab07465e062c831e (127.13.160.130:45027)
I20260925 05:15:41.189606 22131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:45185
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:41.211858 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:41.295682 22461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.295902 22461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.295925 22461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.297024 22461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.297065 22461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.297093 22461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:41.298702 22461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.298966 22461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.299192 22461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:41.304185 22467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.304200 22469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.304199 22466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.304503 22461 server_base.cc:1034] running on GCE node
I20260925 05:15:41.304687 22461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.304951 22461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.306118 22461 hybrid_clock.cc:648] HybridClock initialized: now 1790313341306110 us; error 45 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.309196 22461 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:41.309685 22461 webserver.cc:460] Webserver started at http://127.13.160.131:42231/ using document root <none> and password file <none>
I20260925 05:15:41.309880 22461 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.309924 22461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.310019 22461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:41.311169 22461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "cb16f1ede0214ae881b5cb1f7ba68668"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "780abefd53d55e2176e6756fb853d276"
server_key_iv: "18ed52b3e9715c537dc64f18a452ebc3"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.311492 22461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "cb16f1ede0214ae881b5cb1f7ba68668"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "780abefd53d55e2176e6756fb853d276"
server_key_iv: "18ed52b3e9715c537dc64f18a452ebc3"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.312825 22461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:41.313526 22476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.313707 22461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:41.313791 22461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "cb16f1ede0214ae881b5cb1f7ba68668"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "780abefd53d55e2176e6756fb853d276"
server_key_iv: "18ed52b3e9715c537dc64f18a452ebc3"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.313846 22461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.322791 22461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.324792 22461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.324949 22461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.325177 22461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.325508 22461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.325541 22461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.325564 22461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.325614 22461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.331393 22461 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:41061
I20260925 05:15:41.331501 22589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:41061 every 8 connection(s)
I20260925 05:15:41.331817 22461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:41.336822 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22461
I20260925 05:15:41.336921 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:41.337121 13954 external_mini_cluster.cc:1428] Setting key 522094d779ff740b5ccc5f459279f85c
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:41.341082 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.332575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:49851 (local address 127.13.160.190:45441)
0925 05:15:41.332643 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.332646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.332882 (+ 236us) server_negotiation.cc:407] Connection header received
0925 05:15:41.333348 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.333349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.333359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.333412 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.334309 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.334798 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.335808 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.335987 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.336040 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.338258 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:41.338260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:41.338260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:41.338268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:41.339685 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.340109 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.340111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.340112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.340168 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.340423 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.340424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.340425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.340551 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:41.340623 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:41.340906 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 05:15:41.340987 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:15:41.341303 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.332561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:49851)
0925 05:15:41.332756 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.332792 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.333268 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.333466 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.333469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.334035 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.334233 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.334238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.334878 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.334879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.335734 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.335736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.336062 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.336062 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.336161 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.336644 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.336652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.338142 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:41.339768 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.339775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.339778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.340008 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.340238 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:41.340239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:41.340240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:41.340353 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:41.340626 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:41.340632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:41.340775 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 05:15:41.340884 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 05:15:41.341056 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260925 05:15:41.341807 22590 heartbeater.cc:347] Connected to a master server at 127.13.160.190:45441
I20260925 05:15:41.341918 22590 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:41.342106 22590 heartbeater.cc:511] Master 127.13.160.190:45441 requested a full tablet report, sending...
I20260925 05:15:41.342538 22131 ts_manager.cc:194] Registered new tserver with Master: cb16f1ede0214ae881b5cb1f7ba68668 (127.13.160.131:41061)
I20260925 05:15:41.343101 22131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:49851
I20260925 05:15:41.349359 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
Entry for principal oryx/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab.
WARNING: no policy specified for oryx/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
Entry for principal oryx/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab.
WARNING: no policy specified for oryx/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
Entry for principal oryx/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab.
I20260925 05:15:41.422452 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22193
I20260925 05:15:41.427116 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22327
I20260925 05:15:41.431983 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22461
I20260925 05:15:41.436256 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22099
I20260925 05:15:41.441474 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45441
--webserver_interface=127.13.160.190
--webserver_port=41613
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:45441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:41.522691 22599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.522928 22599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.522954 22599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.524057 22599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:41.524093 22599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.524107 22599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.524120 22599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:41.524132 22599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:41.525570 22599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:45441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:45441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=41613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.525771 22599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.525949 22599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:41.530697 22604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.530697 22607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.530784 22605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.531126 22599 server_base.cc:1034] running on GCE node
I20260925 05:15:41.531399 22599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.531704 22599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.532902 22599 hybrid_clock.cc:648] HybridClock initialized: now 1790313341532894 us; error 37 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.535521 22599 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:41.536046 22599 webserver.cc:460] Webserver started at http://127.13.160.190:41613/ using document root <none> and password file <none>
I20260925 05:15:41.536307 22599 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.536393 22599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.537765 22599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:41.538407 22614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.538579 22599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:41.538691 22599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "e342f5d1456941ca9f5d65bf70f4347d"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "492d6a07aefdd3a54ec6e1a857bb11ee"
server_key_iv: "f2a1b09029fbb54198efd7a212bebfff"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.539093 22599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.550321 22599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.552318 22599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.552493 22599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.556422 22599 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:45441
I20260925 05:15:41.556457 22666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:45441 every 8 connection(s)
I20260925 05:15:41.556857 22599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:41.558372 22667 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:15:41.559064 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Bootstrap starting.
I20260925 05:15:41.566316 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22599
I20260925 05:15:41.566630 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:38375
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:45441
--webserver_port=37571
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:15:41.567464 22667 log.cc:824] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:41.569239 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:15:41.569536 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Bootstrap complete.
I20260925 05:15:41.571903 22667 raft_consensus.cc:348] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:41.572162 22667 raft_consensus.cc:729] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e342f5d1456941ca9f5d65bf70f4347d, State: Initialized, Role: FOLLOWER
I20260925 05:15:41.572386 22667 consensus_queue.cc:260] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:41.572486 22667 raft_consensus.cc:388] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:41.572541 22667 raft_consensus.cc:482] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:41.572590 22667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:15:41.573621 22667 raft_consensus.cc:504] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:41.573745 22667 leader_election.cc:302] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e342f5d1456941ca9f5d65bf70f4347d; no voters:
I20260925 05:15:41.573957 22667 leader_election.cc:288] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 05:15:41.573994 22671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 2 FOLLOWER]: Leader election won for term 2
I20260925 05:15:41.574110 22671 raft_consensus.cc:686] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [term 2 LEADER]: Becoming Leader. State: Replica: e342f5d1456941ca9f5d65bf70f4347d, State: Running, Role: LEADER
I20260925 05:15:41.574198 22671 consensus_queue.cc:237] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } }
I20260925 05:15:41.574247 22667 sys_catalog.cc:564] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:41.574456 22671 sys_catalog.cc:455] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e342f5d1456941ca9f5d65bf70f4347d. Latest consensus state: current_term: 2 leader_uuid: "e342f5d1456941ca9f5d65bf70f4347d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } } }
I20260925 05:15:41.574523 22671 sys_catalog.cc:458] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:41.574481 22673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e342f5d1456941ca9f5d65bf70f4347d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e342f5d1456941ca9f5d65bf70f4347d" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 45441 } } }
I20260925 05:15:41.574692 22673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:41.575382 22675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:41.575958 22675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:41.576115 22675 catalog_manager.cc:1259] Loaded cluster ID: 434dc103b66646d1a8301ad03445d0e5
I20260925 05:15:41.576174 22675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:41.577502 22675 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:41.577689 22675 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e342f5d1456941ca9f5d65bf70f4347d: Loaded TSK: 0
I20260925 05:15:41.577903 22675 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:15:41.651450 22669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.651661 22669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.651687 22669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.652937 22669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.652982 22669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.653009 22669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:41.654471 22669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.129.keytab
--principal=oryx/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:38375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=37571
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.654721 22669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.655005 22669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:41.660995 22693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.661005 22694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.661357 22669 server_base.cc:1034] running on GCE node
W20260925 05:15:41.661015 22696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.661720 22669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.662000 22669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.663172 22669 hybrid_clock.cc:648] HybridClock initialized: now 1790313341663160 us; error 35 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.666000 22669 init.cc:377] Logged in from keytab as oryx/127.13.160.129@KRBTEST.COM (short username oryx)
I20260925 05:15:41.666563 22669 webserver.cc:460] Webserver started at http://127.13.160.129:37571/ using document root <none> and password file <none>
I20260925 05:15:41.666833 22669 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.666956 22669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.668418 22669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:41.669117 22703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.669327 22669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:41.669446 22669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "f65639be70fd4cb5badec2a0afccadab"
format_stamp: "Formatted at 2026-09-25 05:15:40 on dist-test-slave-9tgz"
server_key: "7822436dbd6d1d430e5cb7b717860fff"
server_key_iv: "1ac9b29bed1995c3ea9d17215c99efaa"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.669835 22669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.688544 22669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.690986 22669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.691171 22669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.691450 22669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.691847 22669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.691920 22669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.691967 22669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.691998 22669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.698627 22669 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:38375
I20260925 05:15:41.698676 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:38375 every 8 connection(s)
I20260925 05:15:41.699232 22669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:41.702149 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22669
I20260925 05:15:41.702468 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:45027
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:45441
--webserver_port=44495
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.709659 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.700079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:56299)
0925 05:15:41.700286 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.700333 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.700963 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.701720 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.701725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.702590 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.702877 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.702884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.704484 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.704485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.705475 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.705478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.706364 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.706365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.706465 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.707089 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.707098 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.709504 (+ 2406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
W20260925 05:15:41.709908 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.700164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:56299 (local address 127.13.160.190:45441)
0925 05:15:41.701185 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.701195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.701248 (+ 53us) server_negotiation.cc:407] Connection header received
0925 05:15:41.701410 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.701417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.701472 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.701617 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.703036 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.704368 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.705582 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.706270 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.706400 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.709706 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:56299: BlockingRecv error: recv got EOF from 127.13.160.129:56299 (error 108)
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":76,"threads_started":1}
W20260925 05:15:41.709918 22817 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:45441 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.160.190:45441: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.715133 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.710155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:33021)
0925 05:15:41.710211 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.710224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.710265 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.710415 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.710416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.710584 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.710648 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.710650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.711325 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.711327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.712279 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.712282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.712614 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.712615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.712690 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.713126 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.713132 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.715079 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 05:15:41.715255 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.710220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33021 (local address 127.13.160.190:45441)
0925 05:15:41.710258 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.710260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.710275 (+ 15us) server_negotiation.cc:407] Connection header received
0925 05:15:41.710329 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.710330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.710340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.710390 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.710732 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.711246 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.712364 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.712541 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.712589 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.715190 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:33021: BlockingRecv error: recv got EOF from 127.13.160.129:33021 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.719849 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.715412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:35833)
0925 05:15:41.715577 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.715588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.715640 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.715773 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.715774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.715880 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.715909 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.715910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.716792 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.716792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.717267 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.717269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.717706 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.717706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.717741 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.718172 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.718176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.719793 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
W20260925 05:15:41.719976 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.715521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:35833 (local address 127.13.160.190:45441)
0925 05:15:41.715559 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.715560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.715628 (+ 68us) server_negotiation.cc:407] Connection header received
0925 05:15:41.715692 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.715693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.715703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.715766 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.715975 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.716723 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.717339 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.717636 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.717700 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.719912 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:35833: BlockingRecv error: recv got EOF from 127.13.160.129:35833 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 05:15:41.720005 22817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 05:15:41.788411 22821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.788627 22821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.788652 22821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.789780 22821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.789819 22821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.789845 22821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:41.791251 22821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.130.keytab
--principal=oryx/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:45027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=44495
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.791443 22821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.791620 22821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:41.796603 22828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.796664 22821 server_base.cc:1034] running on GCE node
W20260925 05:15:41.796625 22827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.796602 22830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.797065 22821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.797343 22821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.798491 22821 hybrid_clock.cc:648] HybridClock initialized: now 1790313341798479 us; error 30 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.801070 22821 init.cc:377] Logged in from keytab as oryx/127.13.160.130@KRBTEST.COM (short username oryx)
I20260925 05:15:41.801558 22821 webserver.cc:460] Webserver started at http://127.13.160.130:44495/ using document root <none> and password file <none>
I20260925 05:15:41.801798 22821 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.801872 22821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.803295 22821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:41.803891 22837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.804061 22821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:41.804147 22821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "13fb0c0595c94c8aab07465e062c831e"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "a7d203957f40c35a639b134fa2b3b81e"
server_key_iv: "6633a218066fa4f9c58a7219701f5c7d"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.804483 22821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.828830 22821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.831310 22821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.831425 22821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.831624 22821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.831952 22821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.831988 22821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.832012 22821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.832059 22821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.837935 22821 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:45027
I20260925 05:15:41.837983 22950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:45027 every 8 connection(s)
I20260925 05:15:41.838344 22821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
W20260925 05:15:41.845801 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.839116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:34169 (local address 127.13.160.190:45441)
0925 05:15:41.839192 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.839195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.839432 (+ 237us) server_negotiation.cc:407] Connection header received
0925 05:15:41.839925 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.839925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.839939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.839991 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.840920 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.841404 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.842586 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.842777 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.842827 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.845751 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:34169: BlockingRecv error: recv got EOF from 127.13.160.130:34169 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:15:41.845810 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.839112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:34169)
0925 05:15:41.839305 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.839342 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.839838 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.840047 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.840050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.840620 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.840841 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.840846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.841505 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.841506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.842479 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.842481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.842846 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.842847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.842993 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.843532 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.843541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.845650 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
W20260925 05:15:41.846061 22951 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:45441 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.160.190:45441: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
I20260925 05:15:41.848106 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22821
I20260925 05:15:41.848384 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:41061
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:45441
--webserver_port=42231
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.850629 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.846334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:54633)
0925 05:15:41.846386 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.846412 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.846452 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.846593 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.846594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.846730 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.846772 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.846773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.847452 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.847454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.847964 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.847965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.848359 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.848360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.848420 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.848912 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.848917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.850546 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 05:15:41.850682 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.846376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:54633 (local address 127.13.160.190:45441)
0925 05:15:41.846435 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.846437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.846447 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:41.846514 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.846515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.846531 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.846591 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.846847 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.847372 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.848046 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.848284 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.848363 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.850615 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:54633: BlockingRecv error: recv got EOF from 127.13.160.130:54633 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.855203 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.851148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:41259)
0925 05:15:41.851212 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.851222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.851260 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.851394 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.851394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.851516 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.851541 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.851542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.852149 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.852150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.852636 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.852637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.852989 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.852990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.853043 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.853473 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.853478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.855121 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 05:15:41.855260 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.851233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:41259 (local address 127.13.160.190:45441)
0925 05:15:41.851272 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.851274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.851282 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:41.851305 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.851306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.851315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.851389 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.851620 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.852071 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.852726 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.852910 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.852989 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.855197 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:41259: BlockingRecv error: recv got EOF from 127.13.160.130:41259 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 05:15:41.855602 22951 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 05:15:41.930445 22954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:41.930660 22954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:41.930682 22954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:41.931810 22954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:41.931845 22954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:41.931871 22954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:41.933303 22954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/krb5kdc/oryx_127.13.160.131.keytab
--principal=oryx/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:41061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=42231
--tserver_master_addrs=127.13.160.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:41.933488 22954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:41.933658 22954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:41.938570 22962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.938597 22959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:41.938683 22960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:41.938894 22954 server_base.cc:1034] running on GCE node
I20260925 05:15:41.939101 22954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:41.939294 22954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:41.940434 22954 hybrid_clock.cc:648] HybridClock initialized: now 1790313341940414 us; error 35 us; skew 500 ppm
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313341, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:41.943071 22954 init.cc:377] Logged in from keytab as oryx/127.13.160.131@KRBTEST.COM (short username oryx)
I20260925 05:15:41.943557 22954 webserver.cc:460] Webserver started at http://127.13.160.131:42231/ using document root <none> and password file <none>
I20260925 05:15:41.943804 22954 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:41.943885 22954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:41.945273 22954 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:41.945890 22969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.946067 22954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:41.946167 22954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "cb16f1ede0214ae881b5cb1f7ba68668"
format_stamp: "Formatted at 2026-09-25 05:15:41 on dist-test-slave-9tgz"
server_key: "780abefd53d55e2176e6756fb853d276"
server_key_iv: "18ed52b3e9715c537dc64f18a452ebc3"
server_key_version: "encryptionkey@0"
I20260925 05:15:41.946535 22954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:41.963445 22954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:41.965418 22954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:41.965586 22954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:41.965838 22954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:41.966147 22954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:41.966202 22954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.966235 22954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:41.966265 22954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:41.971840 22954 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:41061
I20260925 05:15:41.971886 23082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:41061 every 8 connection(s)
I20260925 05:15:41.972260 22954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:41.973830 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 22954
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.980795 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.972921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:56703)
0925 05:15:41.973134 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.973183 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.973722 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.973980 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.973984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.974830 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.975092 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.975097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.976081 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.976082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.977306 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.977309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.977711 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.977714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.977821 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.978184 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.978194 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.980644 (+ 2450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":126,"threads_started":1}
W20260925 05:15:41.980804 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.972957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:56703 (local address 127.13.160.190:45441)
0925 05:15:41.973039 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.973041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.973273 (+ 232us) server_negotiation.cc:407] Connection header received
0925 05:15:41.973826 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.973827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.973840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.973908 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.975180 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.975997 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.977378 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.977614 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.977658 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.980748 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:56703: BlockingRecv error: recv got EOF from 127.13.160.131:56703 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 05:15:41.981118 23083 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:45441 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.160.190:45441: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
I20260925 05:15:41.982312 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.974391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.13.160.190:45441)
0925 05:15:41.974502 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.974504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.974513 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:41.974574 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.974575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.974586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.974647 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.975392 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.976069 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.977171 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.977396 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.977458 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.978938 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:41.978943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:41.978943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:41.978955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:41.980591 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.981214 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.981218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.981218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.981391 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:41.981594 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:41.981595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:41.981595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:41.981831 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:41.981987 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:41.982102 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 05:15:41.982216 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.985736 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.981326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:40631)
0925 05:15:41.981370 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.981380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.981437 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.981562 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.981563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.981691 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.981736 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.981737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.982528 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.982529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.983333 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.983337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.983640 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.983640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.983698 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.984112 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.984117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.985692 (+ 1575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 05:15:41.985844 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.981368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:40631 (local address 127.13.160.190:45441)
0925 05:15:41.981436 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.981438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.981447 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:41.981492 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.981493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.981502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.981558 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.981817 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.982343 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.983401 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.983578 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.983631 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.985776 (+ 2145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:40631: BlockingRecv error: recv got EOF from 127.13.160.131:40631 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:41 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:41.989770 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:41.986044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:39583)
0925 05:15:41.986077 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:41.986089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:41.986132 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:41.986267 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:41.986267 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:41.986364 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:41.986389 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.986390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.987019 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.987020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.987476 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:41.987477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.987766 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.987767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:41.987800 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.988219 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:41.988223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:41.989690 (+ 1467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 05:15:41.989820 22820 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:41.986067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:39583 (local address 127.13.160.190:45441)
0925 05:15:41.986104 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:41.986106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:41.986121 (+ 15us) server_negotiation.cc:407] Connection header received
0925 05:15:41.986193 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:41.986194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:41.986203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:41.986251 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:41.986446 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.986957 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.987538 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:41.987703 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:41.987754 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:41.989761 (+ 2007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:39583: BlockingRecv error: recv got EOF from 127.13.160.131:39583 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 05:15:41.989938 23083 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:42.725499 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:42.720474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:38173)
0925 05:15:42.720618 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:42.720630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:42.720708 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:42.720844 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:42.720844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:42.721008 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:42.721058 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.721059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.721779 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.721781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.722542 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.722545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.722953 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.722954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.723011 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.723545 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:42.723556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:42.725427 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260925 05:15:42.725713 23089 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:42.720573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:38173 (local address 127.13.160.190:45441)
0925 05:15:42.720712 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:42.720714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:42.720725 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:42.720753 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:42.720753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:42.720763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:42.720834 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:42.721125 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.721650 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.722640 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.722845 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.722947 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.725566 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:38173: BlockingRecv error: recv got EOF from 127.13.160.129:38173 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":46,"threads_started":1}
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:42.861037 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:42.856077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:44435)
0925 05:15:42.856252 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:42.856266 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:42.856368 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:42.856557 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:42.856558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:42.856775 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:42.856825 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.856827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.857530 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.857531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.858136 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.858139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.858523 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.858524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.858590 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.859086 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:42.859091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:42.860925 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260925 05:15:42.861093 23089 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:42.856187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:44435 (local address 127.13.160.190:45441)
0925 05:15:42.856256 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:42.856259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:42.856346 (+ 87us) server_negotiation.cc:407] Connection header received
0925 05:15:42.856448 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:42.856450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:42.856463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:42.856534 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:42.856909 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.857429 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.858226 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.858429 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.858512 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.861023 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:44435: BlockingRecv error: recv got EOF from 127.13.160.130:44435 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:42 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:42.995218 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:42.990429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:43485)
0925 05:15:42.990580 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:42.990594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:42.990686 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:42.990877 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:42.990879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:42.991088 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:42.991140 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.991142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.991830 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.991831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.992349 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:42.992351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.992687 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.992688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:42.992733 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.993195 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:42.993199 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:42.995104 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":56,"threads_started":1}
W20260925 05:15:42.995260 23089 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:42.990554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:43485 (local address 127.13.160.190:45441)
0925 05:15:42.990633 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:42.990635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:42.990664 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:15:42.990752 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:42.990753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:42.990768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:42.990820 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:42.991246 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.991758 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.992430 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:42.992616 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:42.992673 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:42.995194 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:43485: BlockingRecv error: recv got EOF from 127.13.160.131:43485 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:43.730716 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:43.726057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:52215)
0925 05:15:43.726228 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:43.726241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:43.726300 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:43.726457 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:43.726458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:43.726620 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:43.726671 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.726673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.727336 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.727337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.727869 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.727871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.728207 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.728208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.728246 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:43.728757 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:43.728761 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:43.730644 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
W20260925 05:15:43.730847 23093 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:43.726149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:52215 (local address 127.13.160.190:45441)
0925 05:15:43.726246 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:43.726248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:43.726287 (+ 39us) server_negotiation.cc:407] Connection header received
0925 05:15:43.726357 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:43.726358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:43.726373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:43.726439 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:43.726752 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.727270 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.727944 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.728142 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.728192 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:43.730755 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:52215: BlockingRecv error: recv got EOF from 127.13.160.129:52215 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":46,"threads_started":1}
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:43.866253 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:43.861553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:51663)
0925 05:15:43.861737 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:43.861756 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:43.861817 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:43.861951 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:43.861952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:43.862109 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:43.862160 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.862162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.862820 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.862821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.863446 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.863449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.863777 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.863778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.863819 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:43.864260 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:43.864265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:43.866194 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":104,"threads_started":1}
W20260925 05:15:43.866365 23093 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:43.861643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:51663 (local address 127.13.160.190:45441)
0925 05:15:43.861715 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:43.861717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:43.861814 (+ 97us) server_negotiation.cc:407] Connection header received
0925 05:15:43.861863 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:43.861864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:43.861875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:43.861933 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:43.862249 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.862752 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.863518 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.863712 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.863766 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:43.866292 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:51663: BlockingRecv error: recv got EOF from 127.13.160.130:51663 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:43 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:44.000936 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:43.995887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:46147)
0925 05:15:43.996027 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:43.996039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:43.996130 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:43.996301 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:43.996302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:43.996473 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:43.996524 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.996526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.997463 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.997464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.998197 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:43.998199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.998512 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.998513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:43.998551 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:43.999101 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:43.999106 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:44.000863 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260925 05:15:44.000998 23093 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:43.995955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:46147 (local address 127.13.160.190:45441)
0925 05:15:43.995996 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:43.995999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:43.996126 (+ 127us) server_negotiation.cc:407] Connection header received
0925 05:15:43.996189 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:43.996190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:43.996206 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:43.996255 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:43.996599 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.997390 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.998271 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:43.998447 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:43.998500 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:44.000948 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:46147: BlockingRecv error: recv got EOF from 127.13.160.131:46147 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 05:15:44 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:44 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:44.735806 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:44.731406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:45533)
0925 05:15:44.731569 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:44.731582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:44.731658 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:44.731798 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:44.731798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:44.731958 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:44.732007 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:44.732009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.732636 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.732637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:44.733162 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:44.733163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.733484 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.733484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:44.733524 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:44.733988 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:44.733992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:44.735712 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260925 05:15:44.735940 23097 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:44.731522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:45533 (local address 127.13.160.190:45441)
0925 05:15:44.731657 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:44.731660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:44.731668 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:44.731711 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:44.731712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:44.731721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:44.731778 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:44.732073 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.732567 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.733234 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.733420 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.733469 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:44.735828 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:45533: BlockingRecv error: recv got EOF from 127.13.160.129:45533 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
Sep 25 05:15:44 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:44 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:44.871693 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:44.867014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:54269)
0925 05:15:44.867180 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:44.867192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:44.867257 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:44.867410 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:44.867411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:44.867612 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:44.867657 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:44.867658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.868331 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.868332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:44.868864 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:44.868866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.869238 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.869239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:44.869303 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:44.869771 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:44.869775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:44.871633 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
W20260925 05:15:44.871881 23097 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:44.867130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:54269 (local address 127.13.160.190:45441)
0925 05:15:44.867203 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:44.867225 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:44.867237 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:15:44.867332 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:44.867334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:44.867346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:44.867395 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:44.867735 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.868240 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.868948 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:44.869152 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:44.869237 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:44.871797 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:54269: BlockingRecv error: recv got EOF from 127.13.160.130:54269 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:45.006108 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:45.001466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:60227)
0925 05:15:45.001617 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:45.001627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:45.001685 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:45.001813 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:45.001813 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:45.001969 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:45.002019 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.002020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.002701 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.002701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.003276 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.003278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.003614 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.003614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.003656 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.004154 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:45.004159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:45.005990 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
W20260925 05:15:45.006171 23097 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:45.001521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:60227 (local address 127.13.160.190:45441)
0925 05:15:45.001590 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:45.001592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:45.001686 (+ 94us) server_negotiation.cc:407] Connection header received
0925 05:15:45.001743 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:45.001743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:45.001756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:45.001796 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:45.002117 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.002625 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.003354 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.003535 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.003606 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.006102 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:60227: BlockingRecv error: recv got EOF from 127.13.160.131:60227 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:45.745967 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:45.736294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:46407)
0925 05:15:45.736464 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:45.736475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:45.736531 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:45.736671 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:45.736672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:45.736839 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:45.736891 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.736893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.737547 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.737548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.738095 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.738097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.738452 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.738453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.738493 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.743823 (+ 5330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:45.743830 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:45.745882 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
W20260925 05:15:45.746101 23101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:45.736400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:46407 (local address 127.13.160.190:45441)
0925 05:15:45.736519 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:45.736520 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:45.736529 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:45.736591 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:45.736592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:45.736603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:45.736654 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:45.736960 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.737465 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.738179 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.738381 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.738433 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.746000 (+ 7567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:46407: BlockingRecv error: recv got EOF from 127.13.160.129:46407 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:45 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:45.877354 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:45.872272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:43555)
0925 05:15:45.872477 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:45.872488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:45.872559 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:45.872697 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:45.872698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:45.872868 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:45.872918 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.872920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.873625 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.873626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.874153 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:45.874155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.874589 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.874590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:45.874636 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.875128 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:45.875132 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:45.877293 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":129,"threads_started":1}
W20260925 05:15:45.877496 23101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:45.872374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:43555 (local address 127.13.160.190:45441)
0925 05:15:45.872476 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:45.872478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:45.872558 (+ 80us) server_negotiation.cc:407] Connection header received
0925 05:15:45.872607 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:45.872608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:45.872618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:45.872686 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:45.872981 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.873537 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.874266 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:45.874523 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:45.874577 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:45.877413 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:43555: BlockingRecv error: recv got EOF from 127.13.160.130:43555 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:46.011978 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:46.006824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:36419)
0925 05:15:46.007013 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:46.007025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:46.007132 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:46.007312 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:46.007313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:46.007498 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:46.007547 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.007568 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.008480 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.008482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.009141 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.009144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.009478 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.009479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.009517 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.010047 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:46.010052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:46.011916 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":121,"threads_started":1}
W20260925 05:15:46.012048 23101 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:46.006936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36419 (local address 127.13.160.190:45441)
0925 05:15:46.007002 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:46.007005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:46.007115 (+ 110us) server_negotiation.cc:407] Connection header received
0925 05:15:46.007207 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:46.007208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:46.007228 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:46.007291 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:46.007657 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.008362 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.009226 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.009409 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.009465 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.011998 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:36419: BlockingRecv error: recv got EOF from 127.13.160.131:36419 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:46.751185 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:46.746440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:56119)
0925 05:15:46.746566 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:46.746580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:46.746639 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:46.746794 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:46.746794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:46.746979 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:46.747026 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.747028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.747679 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.747680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.748285 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.748287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.748617 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.748617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.748657 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.749199 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:46.749204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:46.751073 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
W20260925 05:15:46.751235 23105 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:46.746526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:56119 (local address 127.13.160.190:45441)
0925 05:15:46.746670 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:46.746672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:46.746680 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:46.746703 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:46.746704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:46.746714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:46.746779 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:46.747094 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.747591 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.748363 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.748549 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.748598 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.751161 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:56119: BlockingRecv error: recv got EOF from 127.13.160.129:56119 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:46 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:46.882484 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:46.877835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:37549)
0925 05:15:46.877982 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:46.877994 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:46.878054 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:46.878197 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:46.878198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:46.878362 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:46.878411 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.878412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.879143 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.879145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.879689 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:46.879691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.880027 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.880027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:46.880071 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.880540 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:46.880545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:46.882422 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
W20260925 05:15:46.882639 23105 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:46.877976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:37549 (local address 127.13.160.190:45441)
0925 05:15:46.878029 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:46.878031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:46.878042 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:46.878106 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:46.878107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:46.878117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:46.878172 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:46.878486 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.879062 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.879771 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:46.879959 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:46.880010 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:46.882539 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:37549: BlockingRecv error: recv got EOF from 127.13.160.130:37549 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:47.017437 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:47.012560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:43191)
0925 05:15:47.012716 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:47.012728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:47.012795 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:47.012969 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:47.012970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:47.013137 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:47.013190 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.013191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.013863 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.013863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.014400 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.014402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.014750 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.014751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.014791 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.015366 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:47.015371 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:47.017347 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260925 05:15:47.017549 23105 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:47.012640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:43191 (local address 127.13.160.190:45441)
0925 05:15:47.012733 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:47.012735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:47.012848 (+ 113us) server_negotiation.cc:407] Connection header received
0925 05:15:47.012881 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:47.012882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:47.012897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:47.012946 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:47.013255 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.013781 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.014479 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.014682 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.014735 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.017475 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:43191: BlockingRecv error: recv got EOF from 127.13.160.131:43191 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:47.756635 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:47.751723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:43743)
0925 05:15:47.751865 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:47.751885 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:47.751941 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:47.752098 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:47.752099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:47.752265 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:47.752317 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.752318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.752985 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.752986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.753509 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.753511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.754031 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.754032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.754098 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.754714 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:47.754719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:47.756576 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
W20260925 05:15:47.756817 23109 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:47.751825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:43743 (local address 127.13.160.190:45441)
0925 05:15:47.751956 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:47.751958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:47.751967 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:47.751995 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:47.751995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:47.752006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:47.752077 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:47.752388 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.752904 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.753619 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.753935 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.754007 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.756726 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:43743: BlockingRecv error: recv got EOF from 127.13.160.129:43743 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:47 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:47.887980 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:47.883183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:55945)
0925 05:15:47.883320 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:47.883332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:47.883398 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:47.883533 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:47.883533 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:47.883696 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:47.883766 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.883767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.884433 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.884434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.885004 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:47.885006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.885377 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.885377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:47.885423 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.886017 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:47.886022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:47.887919 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260925 05:15:47.888104 23109 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:47.883287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:55945 (local address 127.13.160.190:45441)
0925 05:15:47.883396 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:47.883398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:47.883440 (+ 42us) server_negotiation.cc:407] Connection header received
0925 05:15:47.883469 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:47.883470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:47.883481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:47.883527 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:47.883838 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.884336 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.885097 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:47.885288 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:47.885359 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:47.888023 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:55945: BlockingRecv error: recv got EOF from 127.13.160.130:55945 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:48.022985 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:48.017993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:34993)
0925 05:15:48.018215 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:48.018229 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:48.018302 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:48.018481 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:48.018482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:48.018645 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:48.018712 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.018714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.019387 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.019388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.019933 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.019935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.020309 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.020310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.020371 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.020945 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:48.020951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:48.022866 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
W20260925 05:15:48.023052 23109 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:48.018080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:34993 (local address 127.13.160.190:45441)
0925 05:15:48.018147 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:48.018149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:48.018299 (+ 150us) server_negotiation.cc:407] Connection header received
0925 05:15:48.018387 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:48.018389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:48.018404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:48.018458 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:48.018778 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.019299 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.020022 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.020206 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.020266 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.022989 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:34993: BlockingRecv error: recv got EOF from 127.13.160.131:34993 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:48.761828 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:48.757272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:60671)
0925 05:15:48.757444 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:48.757460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:48.757529 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:48.757678 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:48.757679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:48.757851 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:48.757902 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.757903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.758575 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.758576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.759196 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.759198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.759537 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.759538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.759579 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.760054 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:48.760058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:48.761766 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
W20260925 05:15:48.762013 23113 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:48.757373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:60671 (local address 127.13.160.190:45441)
0925 05:15:48.757515 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:48.757517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:48.757526 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:48.757579 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:48.757580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:48.757591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:48.757651 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:48.757972 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.758484 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.759277 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.759470 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.759522 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.761905 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:60671: BlockingRecv error: recv got EOF from 127.13.160.129:60671 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:48 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:48.893677 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:48.888541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:33271)
0925 05:15:48.888759 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:48.888771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:48.888850 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:48.889020 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:48.889021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:48.889211 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:48.889269 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.889271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.889986 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.889987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.890538 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:48.890540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.891053 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.891055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:48.891111 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.891666 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:48.891671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:48.893616 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
W20260925 05:15:48.893850 23113 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:48.888656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:33271 (local address 127.13.160.190:45441)
0925 05:15:48.888738 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:48.888741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:48.888831 (+ 90us) server_negotiation.cc:407] Connection header received
0925 05:15:48.888912 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:48.888913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:48.888926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:48.888999 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:48.889361 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.889885 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.890637 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:48.890856 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:48.890960 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:48.893782 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:33271: BlockingRecv error: recv got EOF from 127.13.160.130:33271 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:49.028326 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:49.023551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:34835)
0925 05:15:49.023726 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:49.023740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:49.023805 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:49.023952 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:49.023953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:49.024116 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:49.024168 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.024169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.024828 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.024829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.025379 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.025381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.025751 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.025751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.025790 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.026271 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:49.026275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:49.028263 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260925 05:15:49.028476 23113 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:49.023665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:34835 (local address 127.13.160.190:45441)
0925 05:15:49.023744 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:49.023746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:49.023793 (+ 47us) server_negotiation.cc:407] Connection header received
0925 05:15:49.023854 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:49.023855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:49.023867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:49.023933 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:49.024246 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.024751 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.025469 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.025679 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.025736 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.028389 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:34835: BlockingRecv error: recv got EOF from 127.13.160.131:34835 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:49.767114 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:49.762384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:57777)
0925 05:15:49.762527 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:49.762538 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:49.762591 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:49.762818 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:49.762819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:49.763046 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:49.763092 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.763094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.763784 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.763785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.764320 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.764322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.764677 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.764678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.764733 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.765205 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:49.765210 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:49.767046 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
W20260925 05:15:49.767220 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:49.762539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:57777 (local address 127.13.160.190:45441)
0925 05:15:49.762672 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:49.762674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:49.762684 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:49.762720 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:49.762721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:49.762732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:49.762802 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:49.763199 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.763699 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.764410 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.764606 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.764679 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.767145 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:57777: BlockingRecv error: recv got EOF from 127.13.160.129:57777 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:49 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:49.899260 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:49.894307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:49671)
0925 05:15:49.894449 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:49.894461 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:49.894529 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:49.894714 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:49.894716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:49.894990 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:49.895057 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.895060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.895859 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.895860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.896375 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:49.896377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.896701 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.896702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:49.896741 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.897256 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:49.897261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:49.899189 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
W20260925 05:15:49.899408 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:49.894344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:49671 (local address 127.13.160.190:45441)
0925 05:15:49.894418 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:49.894420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:49.894530 (+ 110us) server_negotiation.cc:407] Connection header received
0925 05:15:49.894596 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:49.894596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:49.894610 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:49.894689 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:49.895144 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.895784 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.896451 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:49.896634 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:49.896684 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:49.899325 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:49671: BlockingRecv error: recv got EOF from 127.13.160.130:49671 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:50.034032 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:50.028802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:32857)
0925 05:15:50.028997 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:50.029015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:50.029110 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:50.029239 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:50.029240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:50.029425 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:50.029470 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.029472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.030218 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.030219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.031216 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.031219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.031594 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.031595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.031660 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.032138 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:50.032142 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:50.033971 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
W20260925 05:15:50.034116 23117 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:50.028922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:32857 (local address 127.13.160.190:45441)
0925 05:15:50.028991 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:50.028994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:50.029088 (+ 94us) server_negotiation.cc:407] Connection header received
0925 05:15:50.029170 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:50.029172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:50.029183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:50.029235 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:50.029550 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.030120 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.031304 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.031505 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.031595 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.034065 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:32857: BlockingRecv error: recv got EOF from 127.13.160.131:32857 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:50.778630 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:50.767614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:52597)
0925 05:15:50.767753 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:50.767770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:50.767819 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:50.767966 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:50.767967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:50.768130 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:50.768178 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.768180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.768877 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.768877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.769406 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.769408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.769754 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.769755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.769793 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.776704 (+ 6911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:50.776710 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:50.778571 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
W20260925 05:15:50.778788 23121 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:50.767711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:52597 (local address 127.13.160.190:45441)
0925 05:15:50.767844 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:50.767846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:50.767854 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:50.767875 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:50.767876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:50.767887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:50.767951 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:50.768278 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.768806 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.769486 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.769685 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.769742 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.778700 (+ 8958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:52597: BlockingRecv error: recv got EOF from 127.13.160.129:52597 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:50 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:50.904551 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:50.899797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:43187)
0925 05:15:50.899953 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:50.899964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:50.900033 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:50.900173 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:50.900174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:50.900338 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:50.900387 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.900389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.901029 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.901030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.901545 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:50.901546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.901941 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.901942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:50.901997 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.902537 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:50.902542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:50.904449 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260925 05:15:50.904603 23121 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:50.899886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:43187 (local address 127.13.160.190:45441)
0925 05:15:50.899956 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:50.899958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:50.900019 (+ 61us) server_negotiation.cc:407] Connection header received
0925 05:15:50.900099 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:50.900100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:50.900111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:50.900155 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:50.900456 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.900948 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.901634 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:50.901829 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:50.901886 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:50.904536 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:43187: BlockingRecv error: recv got EOF from 127.13.160.130:43187 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:51.039538 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:51.034620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:58633)
0925 05:15:51.034761 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:51.034773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:51.034839 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:51.034981 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:51.034981 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:51.035241 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:51.035303 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.035306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.036143 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.036144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.036782 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.036785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.037157 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.037158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.037212 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.037644 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:51.037649 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:51.039480 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260925 05:15:51.039788 23121 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:51.034688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:58633 (local address 127.13.160.190:45441)
0925 05:15:51.034765 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:51.034768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:51.034821 (+ 53us) server_negotiation.cc:407] Connection header received
0925 05:15:51.034915 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:51.034917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:51.034929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:51.034977 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:51.035372 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.036021 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.036871 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.037082 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.037158 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.039739 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:58633: BlockingRecv error: recv got EOF from 127.13.160.131:58633 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:51.784245 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:51.779188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:56715)
0925 05:15:51.779328 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:51.779341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:51.779423 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:51.779574 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:51.779575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:51.779751 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:51.779803 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.779804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.780580 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.780580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.781384 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.781388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.781776 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.781777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.781831 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.782313 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:51.782317 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:51.784167 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260925 05:15:51.784349 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:51.779269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:56715 (local address 127.13.160.190:45441)
0925 05:15:51.779428 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:51.779430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:51.779441 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:51.779470 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:51.779471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:51.779482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:51.779548 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:51.779889 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.780481 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.781470 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.781672 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.781741 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.784273 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:56715: BlockingRecv error: recv got EOF from 127.13.160.129:56715 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:51 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:51.910199 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:51.905149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:33453)
0925 05:15:51.905355 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:51.905368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:51.905437 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:51.905572 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:51.905573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:51.905771 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:51.905820 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.905822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.906532 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.906534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.907219 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:51.907221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.907618 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.907619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:51.907673 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.908292 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:51.908297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:51.910096 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":90,"threads_started":1}
W20260925 05:15:51.910280 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:51.905254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:33453 (local address 127.13.160.190:45441)
0925 05:15:51.905348 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:51.905350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:51.905416 (+ 66us) server_negotiation.cc:407] Connection header received
0925 05:15:51.905499 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:51.905500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:51.905514 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:51.905569 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:51.905910 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.906432 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.907311 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:51.907527 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:51.907617 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:51.910201 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:33453: BlockingRecv error: recv got EOF from 127.13.160.130:33453 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:52.051250 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:52.040216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:46223)
0925 05:15:52.040397 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:52.040431 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:52.040526 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:52.040766 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:52.040767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:52.040964 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:52.041018 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.041020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.041808 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.041809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.042359 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.042361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.042775 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.042777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.042848 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.049136 (+ 6288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:52.049143 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:52.051184 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":64,"threads_started":1}
W20260925 05:15:52.051460 23125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:52.040391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:46223 (local address 127.13.160.190:45441)
0925 05:15:52.040463 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:52.040465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:52.040503 (+ 38us) server_negotiation.cc:407] Connection header received
0925 05:15:52.040651 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:52.040652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:52.040669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:52.040757 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:52.041121 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.041673 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.042466 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.042677 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.042772 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.051344 (+ 8572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:46223: BlockingRecv error: recv got EOF from 127.13.160.131:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:52.789644 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:52.784872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:43879)
0925 05:15:52.785045 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:52.785056 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:52.785117 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:52.785284 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:52.785285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:52.785477 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:52.785523 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.785525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.786187 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.786188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.786747 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.786749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.787211 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.787212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.787268 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.787846 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:52.787851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:52.789573 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
W20260925 05:15:52.789733 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:52.784981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:43879 (local address 127.13.160.190:45441)
0925 05:15:52.785095 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:52.785096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:52.785106 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:52.785172 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:52.785179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:52.785190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:52.785257 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:52.785598 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.786109 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.786847 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.787130 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.787196 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.789675 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:43879: BlockingRecv error: recv got EOF from 127.13.160.129:43879 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:52 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:52.915860 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:52.910757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:60031)
0925 05:15:52.910966 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:52.910978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:52.911079 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:52.911264 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:52.911265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:52.911465 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:52.911510 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.911511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.912351 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.912352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.912971 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:52.912973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.913358 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.913358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:52.913400 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.913880 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:52.913884 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:52.915788 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
W20260925 05:15:52.916006 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:52.910860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:60031 (local address 127.13.160.190:45441)
0925 05:15:52.910941 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:52.910943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:52.911060 (+ 117us) server_negotiation.cc:407] Connection header received
0925 05:15:52.911176 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:52.911177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:52.911189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:52.911255 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:52.911630 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.912249 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.913084 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:52.913281 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:52.913344 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:52.915919 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:60031: BlockingRecv error: recv got EOF from 127.13.160.130:60031 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:53.056984 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:53.051762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:54539)
0925 05:15:53.051977 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:53.051988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:53.052088 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:53.052270 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:53.052271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:53.052469 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:53.052517 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.052519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.053219 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.053220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.053767 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.053769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.054162 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.054163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.054221 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.054774 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:53.054781 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:53.056907 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
W20260925 05:15:53.057101 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:53.051860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:54539 (local address 127.13.160.190:45441)
0925 05:15:53.051956 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:53.051958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:53.052066 (+ 108us) server_negotiation.cc:407] Connection header received
0925 05:15:53.052164 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:53.052165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:53.052185 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:53.052260 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:53.052613 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.053112 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.053871 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.054067 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.054160 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.057022 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:54539: BlockingRecv error: recv got EOF from 127.13.160.131:54539 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:53.795025 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:53.790259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:33849)
0925 05:15:53.790424 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:53.790436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:53.790500 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:53.790662 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:53.790663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:53.790830 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:53.790876 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.790877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.791601 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.791602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.792392 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.792394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.792723 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.792723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.792761 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.793195 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:53.793200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:53.794967 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
W20260925 05:15:53.795110 23133 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:53.790382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33849 (local address 127.13.160.190:45441)
0925 05:15:53.790523 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:53.790525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:53.790534 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:53.790559 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:53.790560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:53.790571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:53.790646 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:53.791015 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.791519 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.792458 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.792658 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.792712 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.795025 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:33849: BlockingRecv error: recv got EOF from 127.13.160.129:33849 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:53 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:53.921018 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:53.916302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:55587)
0925 05:15:53.916443 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:53.916473 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:53.916526 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:53.916780 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:53.916780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:53.916948 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:53.917004 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.917006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.917661 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.917662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.918202 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:53.918204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.918542 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.918543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:53.918582 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.919129 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:53.919134 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:53.920911 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":99,"threads_started":1}
W20260925 05:15:53.921054 23133 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:53.916570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:55587 (local address 127.13.160.190:45441)
0925 05:15:53.916647 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:53.916648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:53.916655 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:15:53.916687 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:53.916688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:53.916699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:53.916751 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:53.917077 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.917564 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.918283 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:53.918475 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:53.918530 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:53.920990 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:55587: BlockingRecv error: recv got EOF from 127.13.160.130:55587 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:54.062275 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:54.057531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:49845)
0925 05:15:54.057743 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:54.057756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:54.057832 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:54.057985 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:54.057986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:54.058192 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:54.058237 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.058239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.058943 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.058944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.059478 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.059479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.059849 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.059850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.059908 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.060416 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:54.060421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:54.062175 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":109,"threads_started":1}
W20260925 05:15:54.062318 23133 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:54.057652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:49845 (local address 127.13.160.190:45441)
0925 05:15:54.057727 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:54.057730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:54.057810 (+ 80us) server_negotiation.cc:407] Connection header received
0925 05:15:54.057890 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:54.057891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:54.057907 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:54.057979 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:54.058317 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.058819 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.059563 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.059761 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.059849 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.062254 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:49845: BlockingRecv error: recv got EOF from 127.13.160.131:49845 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:54.799834 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:54.795487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:33157)
0925 05:15:54.795638 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:54.795649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:54.795699 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:54.795824 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:54.795825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:54.796017 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:54.796063 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.796065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.796696 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.796696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.797238 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.797239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.797564 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.797565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.797604 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.798059 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:54.798063 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:54.799766 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
W20260925 05:15:54.799968 23137 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:54.795560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33157 (local address 127.13.160.190:45441)
0925 05:15:54.795674 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:54.795675 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:54.795686 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:54.795751 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:54.795752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:54.795763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:54.795819 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:54.796131 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.796628 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.797311 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.797500 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.797549 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.799860 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:33157: BlockingRecv error: recv got EOF from 127.13.160.129:33157 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:54 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:54.926188 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:54.921614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:33233)
0925 05:15:54.921760 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:54.921772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:54.921832 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:54.921979 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:54.921980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:54.922164 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:54.922217 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.922219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.922859 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.922860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.923454 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:54.923456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.923792 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.923792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:54.923832 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.924270 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:54.924274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:54.926125 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
W20260925 05:15:54.926299 23137 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:54.921791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:33233 (local address 127.13.160.190:45441)
0925 05:15:54.921856 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:54.921858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:54.921869 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:15:54.921898 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:54.921899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:54.921910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:54.921959 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:54.922286 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.922771 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.923525 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:54.923726 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:54.923777 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:54.926225 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:33233: BlockingRecv error: recv got EOF from 127.13.160.130:33233 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:55.067251 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:55.062887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:41427)
0925 05:15:55.063082 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:55.063095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:55.063144 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:55.063283 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:55.063284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:55.063446 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:55.063494 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.063496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.064133 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.064134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.064637 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.064639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.064956 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.064957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.064994 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.065455 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:55.065459 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:55.067191 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
W20260925 05:15:55.067373 23137 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:55.063047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:41427 (local address 127.13.160.190:45441)
0925 05:15:55.063123 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:55.063126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:55.063164 (+ 38us) server_negotiation.cc:407] Connection header received
0925 05:15:55.063196 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:55.063197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:55.063211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:55.063264 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:55.063559 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.064054 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.064709 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.064891 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.064948 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.067288 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:41427: BlockingRecv error: recv got EOF from 127.13.160.131:41427 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:55.810534 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:55.800327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:38979)
0925 05:15:55.800480 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:55.800494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:55.800565 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:55.800719 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:55.800720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:55.800888 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:55.800935 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.800937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.801715 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.801716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.802299 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.802301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.802662 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.802663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.802706 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.808686 (+ 5980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:55.808693 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:55.810474 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
W20260925 05:15:55.810667 23141 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:55.800386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:38979 (local address 127.13.160.190:45441)
0925 05:15:55.800534 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:55.800535 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:55.800548 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:15:55.800625 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:55.800625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:55.800635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:55.800695 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:55.801030 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.801547 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.802389 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.802582 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.802643 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.810578 (+ 7935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:38979: BlockingRecv error: recv got EOF from 127.13.160.129:38979 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:55 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:55.931728 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:55.926784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:34767)
0925 05:15:55.926954 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:55.926972 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:55.927048 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:55.927215 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:55.927216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:55.927411 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:55.927456 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.927459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.928134 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.928135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.928687 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:55.928689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.929084 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.929085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:55.929146 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.929752 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:55.929758 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:55.931621 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
W20260925 05:15:55.931789 23141 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:55.926991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:34767 (local address 127.13.160.190:45441)
0925 05:15:55.927071 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:55.927073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:55.927083 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:55.927116 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:55.927126 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:55.927138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:55.927208 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:55.927541 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.928049 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.928780 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:55.928999 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:55.929082 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:55.931712 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:34767: BlockingRecv error: recv got EOF from 127.13.160.130:34767 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.131@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:56.072965 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:56.067883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.131:35675)
0925 05:15:56.068016 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:56.068028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:56.068085 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:56.068298 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:56.068299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:56.068484 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:56.068531 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.068533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.069210 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.069212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.069759 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.069761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.070120 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.070121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.070185 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.070863 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:56.070869 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:56.072907 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260925 05:15:56.073064 23141 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:56.068080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:35675 (local address 127.13.160.190:45441)
0925 05:15:56.068164 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:56.068166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:56.068174 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:56.068205 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:56.068205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:56.068218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:56.068283 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:56.068618 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.069122 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.069853 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.070046 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.070123 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.072997 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.131:35675: BlockingRecv error: recv got EOF from 127.13.160.131:35675 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.129@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:56.815749 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:56.811145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.129:43451)
0925 05:15:56.811305 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:56.811316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:56.811372 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:56.811515 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:56.811515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:56.811677 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:56.811727 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.811728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.812361 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.812365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.812958 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.812960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.813380 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.813381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.813436 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.813867 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:56.813871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:56.815687 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
W20260925 05:15:56.815847 23145 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:56.811237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:43451 (local address 127.13.160.190:45441)
0925 05:15:56.811354 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:56.811356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:56.811364 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:15:56.811432 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:56.811433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:56.811443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:56.811493 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:56.811794 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.812294 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.813046 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.813294 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.813369 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.815776 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.129:43451: BlockingRecv error: recv got EOF from 127.13.160.129:43451 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
Sep 25 05:15:56 dist-test-slave-9tgz krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.160.130@KRBTEST.COM for oryx/127.13.160.190@KRBTEST.COM, Server not found in Kerberos database
I20260925 05:15:56.936793 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:56.932290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:45441 (local address 127.13.160.130:53359)
0925 05:15:56.932435 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:56.932446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:56.932503 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:56.932716 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:56.932716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:56.932874 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:56.932921 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.932923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.933641 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.933642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.934150 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:56.934152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.934462 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.934463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:56.934500 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.935031 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:56.935043 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:56.936736 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.160.190:45441: Server oryx/127.13.160.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
W20260925 05:15:56.936913 23145 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:56.932518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:53359 (local address 127.13.160.190:45441)
0925 05:15:56.932584 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:56.932590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:56.932600 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:15:56.932625 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:56.932626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:56.932636 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:56.932699 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:56.932986 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.933572 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.934223 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:56.934401 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:56.934450 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:56.936840 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.160.130:53359: BlockingRecv error: recv got EOF from 127.13.160.130:53359 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:15:56.974363 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22669
I20260925 05:15:56.978775 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22821
I20260925 05:15:56.983073 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22954
I20260925 05:15:56.987560 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 22599
2026-09-25T05:15:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17444 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T05:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:57Z Disabled control of system clock
I20260925 05:15:57.015100 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35487
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:35487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 05:15:57.095186 23150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.095391 23150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.095412 23150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.096529 23150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:57.096562 23150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.096575 23150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.096587 23150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:57.096599 23150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:57.097927 23150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:35487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.098120 23150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.098363 23150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.103173 23158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.103156 23155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.103156 23156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.103229 23150 server_base.cc:1034] running on GCE node
I20260925 05:15:57.103623 23150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.103948 23150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.105119 23150 hybrid_clock.cc:648] HybridClock initialized: now 1790313357105105 us; error 45 us; skew 500 ppm
I20260925 05:15:57.106340 23150 webserver.cc:460] Webserver started at http://127.13.160.190:33957/ using document root <none> and password file <none>
I20260925 05:15:57.106602 23150 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.106678 23150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.106835 23150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.107945 23150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "cbb1e0d077bb43b093be695c7e19bbfe"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "ecc9e20dfb5deba1253155c46e73e6dc"
server_key_iv: "4b639f2ee98eb839a1e510a0864228a8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.108314 23150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "cbb1e0d077bb43b093be695c7e19bbfe"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "ecc9e20dfb5deba1253155c46e73e6dc"
server_key_iv: "4b639f2ee98eb839a1e510a0864228a8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.109611 23150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:57.110360 23164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.110546 23150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:57.110638 23150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "cbb1e0d077bb43b093be695c7e19bbfe"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "ecc9e20dfb5deba1253155c46e73e6dc"
server_key_iv: "4b639f2ee98eb839a1e510a0864228a8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.110723 23150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.124935 23150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.125255 23150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.125432 23150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.129263 23150 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:35487
I20260925 05:15:57.129315 23216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:35487 every 8 connection(s)
I20260925 05:15:57.129688 23150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:57.130385 23217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:57.133028 23217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe: Bootstrap starting.
I20260925 05:15:57.133576 23217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:57.133919 23217 log.cc:824] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:57.134631 23217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe: No bootstrap required, opened a new log
I20260925 05:15:57.136251 23217 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } }
I20260925 05:15:57.136401 23217 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:57.136425 23217 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb1e0d077bb43b093be695c7e19bbfe, State: Initialized, Role: FOLLOWER
I20260925 05:15:57.136581 23217 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } }
I20260925 05:15:57.136674 23217 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:57.136699 23217 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:57.136729 23217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:57.137270 23217 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } }
I20260925 05:15:57.137356 23217 leader_election.cc:302] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbb1e0d077bb43b093be695c7e19bbfe; no voters:
I20260925 05:15:57.137488 23217 leader_election.cc:288] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:57.137570 23220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:57.137745 23220 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [term 1 LEADER]: Becoming Leader. State: Replica: cbb1e0d077bb43b093be695c7e19bbfe, State: Running, Role: LEADER
I20260925 05:15:57.137799 23217 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:57.137846 23220 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } }
I20260925 05:15:57.138237 23222 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb1e0d077bb43b093be695c7e19bbfe. Latest consensus state: current_term: 1 leader_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } } }
I20260925 05:15:57.138325 23222 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:57.138368 23221 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb1e0d077bb43b093be695c7e19bbfe" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35487 } } }
I20260925 05:15:57.138430 23221 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:57.139082 23227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:57.139098 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23150
I20260925 05:15:57.139187 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:57.139374 13954 external_mini_cluster.cc:1428] Setting key c6e3c827d177c18b0f1b7fee4459ccf6
I20260925 05:15:57.139931 23227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:57.141616 23227 catalog_manager.cc:1347] Generated new cluster ID: 7061b73e1cb741d387d22dfdc6a9b2cf
I20260925 05:15:57.141667 23227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:57.146063 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.140082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.13.160.190:35487)
0925 05:15:57.140269 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.140280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.140313 (+ 33us) server_negotiation.cc:407] Connection header received
0925 05:15:57.140634 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.140642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.140725 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.140877 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.141091 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.142452 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.143541 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.144191 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.144296 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.145233 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.145238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.145238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.145463 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.145506 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.145520 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.145575 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.145763 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.145844 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":84,"threads_started":1}
I20260925 05:15:57.166139 23227 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:57.166712 23227 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:57.171564 23227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbb1e0d077bb43b093be695c7e19bbfe: Generated new TSK 0
I20260925 05:15:57.171809 23227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:57.176985 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--builtin_ntp_servers=127.13.160.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 05:15:57.270043 23241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.270323 23241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.270359 23241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.271627 23241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.271713 23241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.271762 23241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:57.273307 23241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.273602 23241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.273926 23241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.280061 23247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.280115 23241 server_base.cc:1034] running on GCE node
W20260925 05:15:57.280025 23246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.280030 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.280524 23241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.280797 23241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.281975 23241 hybrid_clock.cc:648] HybridClock initialized: now 1790313357281961 us; error 31 us; skew 500 ppm
I20260925 05:15:57.283445 23241 webserver.cc:460] Webserver started at http://127.13.160.129:44013/ using document root <none> and password file <none>
I20260925 05:15:57.283695 23241 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.283783 23241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.283936 23241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.285060 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "87f1240739d54bd389f568864a181be6"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "fdee991cd6d26754c72f0079a1496499"
server_key_iv: "0ef63d99f9320e5898101a3cbcebb5a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.285454 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "87f1240739d54bd389f568864a181be6"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "fdee991cd6d26754c72f0079a1496499"
server_key_iv: "0ef63d99f9320e5898101a3cbcebb5a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.286826 23241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:57.287639 23255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.287824 23241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:57.287935 23241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "87f1240739d54bd389f568864a181be6"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "fdee991cd6d26754c72f0079a1496499"
server_key_iv: "0ef63d99f9320e5898101a3cbcebb5a1"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.288017 23241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.300032 23241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.300376 23241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.300531 23241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.300778 23241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:57.301124 23241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:57.301180 23241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.301230 23241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:57.301254 23241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.307660 23241 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:44677
I20260925 05:15:57.307708 23368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:44677 every 8 connection(s)
I20260925 05:15:57.308089 23241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:57.313141 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23241
I20260925 05:15:57.313274 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:57.313485 13954 external_mini_cluster.cc:1428] Setting key d7c4b336fcf84d7eed052a538b634eb3
I20260925 05:15:57.314586 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--builtin_ntp_servers=127.13.160.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 05:15:57.315555 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.308902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:55661 (local address 127.13.160.190:35487)
0925 05:15:57.308972 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.308974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.309297 (+ 323us) server_negotiation.cc:407] Connection header received
0925 05:15:57.309794 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.309795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.309809 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.309867 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.310542 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.311168 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.312763 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.313079 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.313174 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.315235 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.315236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.315237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.315244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.315263 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.315269 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.315335 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.315392 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.315437 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:15:57.315727 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.308932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35487 (local address 127.13.160.129:55661)
0925 05:15:57.309153 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:57.309192 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:57.309701 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:57.309929 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:57.309932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:57.309941 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:57.310458 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.310463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.311264 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.311265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.312644 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.312648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.313261 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.313262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.313447 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.315041 (+ 1594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:57.315042 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:57.315103 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:57.315140 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:57.315337 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:57.315340 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:57.315343 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:57.315394 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:57.315489 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260925 05:15:57.316212 23369 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35487
I20260925 05:15:57.316339 23369 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:57.316535 23369 heartbeater.cc:511] Master 127.13.160.190:35487 requested a full tablet report, sending...
I20260925 05:15:57.317118 23181 ts_manager.cc:194] Registered new tserver with Master: 87f1240739d54bd389f568864a181be6 (127.13.160.129:44677)
I20260925 05:15:57.317754 23181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.129:55661
W20260925 05:15:57.401968 23372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.402194 23372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.402225 23372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.403596 23372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.403648 23372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.403677 23372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:57.405139 23372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.405448 23372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.405776 23372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.411082 23377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.411080 23378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.411141 23380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.411319 23372 server_base.cc:1034] running on GCE node
I20260925 05:15:57.411545 23372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.411765 23372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.412910 23372 hybrid_clock.cc:648] HybridClock initialized: now 1790313357412900 us; error 28 us; skew 500 ppm
I20260925 05:15:57.414237 23372 webserver.cc:460] Webserver started at http://127.13.160.130:46513/ using document root <none> and password file <none>
I20260925 05:15:57.414503 23372 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.414575 23372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.414732 23372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.415932 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "9a36ff5cbc3749c5b0ea89d87cedd577"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "9bba75563b3f419ef2d585322ea33686"
server_key_iv: "8b5ca298705b244f73e677fa2040e563"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.416312 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "9a36ff5cbc3749c5b0ea89d87cedd577"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "9bba75563b3f419ef2d585322ea33686"
server_key_iv: "8b5ca298705b244f73e677fa2040e563"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.417685 23372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:57.418457 23386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.418635 23372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:57.418754 23372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "9a36ff5cbc3749c5b0ea89d87cedd577"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "9bba75563b3f419ef2d585322ea33686"
server_key_iv: "8b5ca298705b244f73e677fa2040e563"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.418834 23372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.433125 23372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.433483 23372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.433635 23372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.433882 23372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:57.434229 23372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:57.434302 23372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.434343 23372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:57.434370 23372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.440547 23372 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:44223
I20260925 05:15:57.440616 23499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:44223 every 8 connection(s)
I20260925 05:15:57.440996 23372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:57.446615 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.441817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:37295 (local address 127.13.160.190:35487)
0925 05:15:57.441890 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.441892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.442158 (+ 266us) server_negotiation.cc:407] Connection header received
0925 05:15:57.442747 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.442748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.442761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.442816 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.443598 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.444129 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.445146 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.445375 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.445454 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.446298 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.446300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.446300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.446306 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.446321 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.446325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.446394 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.446499 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.446532 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 05:15:57.446857 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.441808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35487 (local address 127.13.160.130:37295)
0925 05:15:57.442027 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:57.442064 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:57.442622 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:57.442929 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:57.442933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:57.442945 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:57.443492 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.443496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.444239 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.444240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.445048 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.445050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.445462 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.445463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.445608 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.446144 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:57.446144 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:57.446174 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:57.446198 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:57.446397 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:57.446403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:57.446408 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:15:57.446489 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 05:15:57.446597 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260925 05:15:57.447451 23500 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35487
I20260925 05:15:57.447556 23500 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:57.447721 23500 heartbeater.cc:511] Master 127.13.160.190:35487 requested a full tablet report, sending...
I20260925 05:15:57.448112 23181 ts_manager.cc:194] Registered new tserver with Master: 9a36ff5cbc3749c5b0ea89d87cedd577 (127.13.160.130:44223)
I20260925 05:15:57.448503 23181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.130:37295
I20260925 05:15:57.450727 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23372
I20260925 05:15:57.450807 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:57.451025 13954 external_mini_cluster.cc:1428] Setting key b1905f7c11156bb4d8ffaf1804891cac
I20260925 05:15:57.451990 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--builtin_ntp_servers=127.13.160.148:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 05:15:57.536872 23503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.537139 23503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.537177 23503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.538532 23503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.538626 23503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.538674 23503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:57.540177 23503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.540401 23503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.540675 23503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.545864 23509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.545892 23508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.545864 23511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.546169 23503 server_base.cc:1034] running on GCE node
I20260925 05:15:57.546388 23503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.546649 23503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.547818 23503 hybrid_clock.cc:648] HybridClock initialized: now 1790313357547789 us; error 62 us; skew 500 ppm
I20260925 05:15:57.549106 23503 webserver.cc:460] Webserver started at http://127.13.160.131:38099/ using document root <none> and password file <none>
I20260925 05:15:57.549379 23503 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.549475 23503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.549645 23503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.550776 23503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "ac24500843be402dade71fdc05a5b3c5"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "0fbab6e1588519675a215d4109fb920b"
server_key_iv: "e4f75315d9b0607ddb37bbece73c5ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.551242 23503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "ac24500843be402dade71fdc05a5b3c5"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "0fbab6e1588519675a215d4109fb920b"
server_key_iv: "e4f75315d9b0607ddb37bbece73c5ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.552629 23503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:57.553377 23517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.553575 23503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:57.553678 23503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "ac24500843be402dade71fdc05a5b3c5"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "0fbab6e1588519675a215d4109fb920b"
server_key_iv: "e4f75315d9b0607ddb37bbece73c5ed8"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.553768 23503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.566283 23503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.566722 23503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.566936 23503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.567222 23503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:57.567612 23503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:57.567669 23503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.567705 23503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:57.567735 23503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.573760 23503 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:34971
I20260925 05:15:57.573830 23630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:34971 every 8 connection(s)
I20260925 05:15:57.574239 23503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:57.576905 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23503
I20260925 05:15:57.577019 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:57.577193 13954 external_mini_cluster.cc:1428] Setting key 25909ccb72af334d700b776b23d1b821
I20260925 05:15:57.580090 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.575145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:35967 (local address 127.13.160.190:35487)
0925 05:15:57.575235 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.575238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.575469 (+ 231us) server_negotiation.cc:407] Connection header received
0925 05:15:57.575966 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.575968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.575979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.576034 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.576721 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.577479 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.578563 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.578788 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.578865 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.579780 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.579781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.579782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.579787 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.579804 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.579808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.579863 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.579966 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.580004 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 05:15:57.580332 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.575120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35487 (local address 127.13.160.131:35967)
0925 05:15:57.575343 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:57.575378 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:57.575880 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:57.576098 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:57.576102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:57.576111 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:57.576627 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.576631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.577596 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.577597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.578454 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:57.578457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:57.578885 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:57.578886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:57.579070 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:57.579601 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:57.579601 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:57.579632 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:57.579656 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:57.579868 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:57.579874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:57.579877 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:57.579970 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 05:15:57.580082 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260925 05:15:57.580794 23631 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35487
I20260925 05:15:57.580924 23631 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:57.581122 23631 heartbeater.cc:511] Master 127.13.160.190:35487 requested a full tablet report, sending...
I20260925 05:15:57.581526 23181 ts_manager.cc:194] Registered new tserver with Master: ac24500843be402dade71fdc05a5b3c5 (127.13.160.131:34971)
I20260925 05:15:57.581904 23181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.131:35967
I20260925 05:15:57.589054 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 05:15:57.591269 23239 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:57.590577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.13.160.190:35487)
0925 05:15:57.590657 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.590659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.590773 (+ 114us) server_negotiation.cc:407] Connection header received
0925 05:15:57.590871 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.590873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.590887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.591002 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.591160 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38270: BlockingRecv error: recv got EOF from 127.0.0.1:38270 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 05:15:57.591346 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23241
I20260925 05:15:57.596066 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23372
I20260925 05:15:57.600235 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23503
I20260925 05:15:57.604803 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23150
2026-09-25T05:15:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (616 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T05:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:57Z Disabled control of system clock
I20260925 05:15:57.631860 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36545
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:36545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 05:15:57.714883 23642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.715153 23642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.715178 23642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.716486 23642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:57.716535 23642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.716548 23642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.716562 23642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:57.716574 23642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:57.717971 23642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:36545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.718199 23642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.718468 23642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:15:57.723522 23642 server_base.cc:1034] running on GCE node
W20260925 05:15:57.723440 23647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.723444 23650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.723440 23648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.724052 23642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.724413 23642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.725589 23642 hybrid_clock.cc:648] HybridClock initialized: now 1790313357725569 us; error 58 us; skew 500 ppm
I20260925 05:15:57.727084 23642 webserver.cc:460] Webserver started at http://127.13.160.190:46553/ using document root <none> and password file <none>
I20260925 05:15:57.727372 23642 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.727448 23642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.727604 23642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.728739 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "507914bff7804e799c71725d293dbb07"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "eefc9970aa9343bd7e0b9dd9c1f2678d"
server_key_iv: "b54b961d8b0234019ffb8721384baf62"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.729123 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "507914bff7804e799c71725d293dbb07"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "eefc9970aa9343bd7e0b9dd9c1f2678d"
server_key_iv: "b54b961d8b0234019ffb8721384baf62"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.730540 23642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:57.731406 23656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.731581 23642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.731696 23642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "507914bff7804e799c71725d293dbb07"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "eefc9970aa9343bd7e0b9dd9c1f2678d"
server_key_iv: "b54b961d8b0234019ffb8721384baf62"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.731782 23642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.735471 23642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.735785 23642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.735942 23642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.740114 23642 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:36545
I20260925 05:15:57.740163 23708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:36545 every 8 connection(s)
I20260925 05:15:57.740579 23642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:57.741317 23709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:57.744554 23709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07: Bootstrap starting.
I20260925 05:15:57.745263 23709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:57.745613 23709 log.cc:824] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:57.746434 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23642
I20260925 05:15:57.746459 23709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07: No bootstrap required, opened a new log
I20260925 05:15:57.746544 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:57.746745 13954 external_mini_cluster.cc:1428] Setting key c4d6b35a80b969975421b7f3ebd84da7
I20260925 05:15:57.748881 23709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } }
I20260925 05:15:57.749089 23709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:57.749135 23709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507914bff7804e799c71725d293dbb07, State: Initialized, Role: FOLLOWER
I20260925 05:15:57.749306 23709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } }
I20260925 05:15:57.749400 23709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:57.749440 23709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:57.749524 23709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:57.750401 23709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } }
I20260925 05:15:57.750540 23709 leader_election.cc:302] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 507914bff7804e799c71725d293dbb07; no voters:
I20260925 05:15:57.750746 23709 leader_election.cc:288] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:57.750773 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.747385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.13.160.190:36545)
0925 05:15:57.747622 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.747632 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.747668 (+ 36us) server_negotiation.cc:407] Connection header received
0925 05:15:57.748824 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.748835 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.748895 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.749081 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.749861 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.749865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.749866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.749871 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.749927 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.749942 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.750030 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.750089 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.750163 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260925 05:15:57.750820 23714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:57.751008 23714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [term 1 LEADER]: Becoming Leader. State: Replica: 507914bff7804e799c71725d293dbb07, State: Running, Role: LEADER
I20260925 05:15:57.751053 23709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:57.751117 23714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } }
I20260925 05:15:57.751492 23716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 507914bff7804e799c71725d293dbb07. Latest consensus state: current_term: 1 leader_uuid: "507914bff7804e799c71725d293dbb07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } } }
I20260925 05:15:57.751595 23715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "507914bff7804e799c71725d293dbb07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507914bff7804e799c71725d293dbb07" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36545 } } }
I20260925 05:15:57.751680 23715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:57.751670 23716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:57.752488 23722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:57.753074 23722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:57.754554 23722 catalog_manager.cc:1347] Generated new cluster ID: a03c1c707ea84638b35a51cecfb5fe26
I20260925 05:15:57.754598 23722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:15:57.763046 23722 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:57.763265 23722 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:57.778267 23722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 507914bff7804e799c71725d293dbb07: Generated new TSK 0
I20260925 05:15:57.778520 23722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:15:57.785593 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--builtin_ntp_servers=127.13.160.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 05:15:57.870502 23733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.870724 23733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.870748 23733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.872033 23733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.872085 23733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.872113 23733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:57.873536 23733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.873814 23733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.874150 23733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.879344 23738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.879385 23739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.879344 23741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.879558 23733 server_base.cc:1034] running on GCE node
I20260925 05:15:57.879830 23733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.880131 23733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.881294 23733 hybrid_clock.cc:648] HybridClock initialized: now 1790313357881271 us; error 42 us; skew 500 ppm
I20260925 05:15:57.882654 23733 webserver.cc:460] Webserver started at http://127.13.160.129:35545/ using document root <none> and password file <none>
I20260925 05:15:57.882999 23733 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.883054 23733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.883188 23733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.884331 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "f10c824aaf314357a275951e7c3ceab2"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "6e9a99157eea1a1d598ce3c0ba2c232c"
server_key_iv: "ac1366d6fb7179f8cbca3dd52c0a710e"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.884711 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "f10c824aaf314357a275951e7c3ceab2"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "6e9a99157eea1a1d598ce3c0ba2c232c"
server_key_iv: "ac1366d6fb7179f8cbca3dd52c0a710e"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.886184 23733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:15:57.887037 23747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.887226 23733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:57.887331 23733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "f10c824aaf314357a275951e7c3ceab2"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "6e9a99157eea1a1d598ce3c0ba2c232c"
server_key_iv: "ac1366d6fb7179f8cbca3dd52c0a710e"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.887396 23733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:57.890837 23733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:57.891168 23733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:57.891332 23733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:57.891561 23733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:57.891925 23733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:57.891961 23733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.892026 23733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:57.892060 23733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:57.898262 23733 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:41853
I20260925 05:15:57.898396 23860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:41853 every 8 connection(s)
I20260925 05:15:57.898736 23733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:15:57.900851 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23733
I20260925 05:15:57.900951 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:57.901146 13954 external_mini_cluster.cc:1428] Setting key 44b0b33f54c0303773a6c9ea90060906
I20260925 05:15:57.902208 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--builtin_ntp_servers=127.13.160.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 05:15:57.902277 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.899588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:49419 (local address 127.13.160.190:36545)
0925 05:15:57.899662 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:57.899664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:57.899990 (+ 326us) server_negotiation.cc:407] Connection header received
0925 05:15:57.900464 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:57.900465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:57.900472 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:57.900541 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:57.902032 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:57.902034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:57.902034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:57.902038 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:57.902054 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:57.902058 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:57.902111 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:57.902210 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 05:15:57.902222 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:15:57.902762 23863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:57.899599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36545 (local address 127.13.160.129:49419)
0925 05:15:57.899822 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:57.899886 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:57.900374 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:57.900598 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:57.900602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:57.900611 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:57.901850 (+ 1239us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:57.901851 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:57.901892 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:57.901931 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:57.902122 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:57.902129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:57.902132 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:57.902224 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 05:15:57.902507 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260925 05:15:57.903386 23861 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36545
I20260925 05:15:57.903499 23861 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:57.903671 23861 heartbeater.cc:511] Master 127.13.160.190:36545 requested a full tablet report, sending...
I20260925 05:15:57.904295 23673 ts_manager.cc:194] Registered new tserver with Master: f10c824aaf314357a275951e7c3ceab2 (127.13.160.129:41853)
W20260925 05:15:57.985662 23864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:57.985894 23864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:57.985918 23864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:57.987278 23864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:57.987334 23864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:57.987367 23864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:57.988865 23864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:57.989073 23864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:57.989331 23864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:57.994405 23872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.994410 23869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:57.994410 23870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:57.994799 23864 server_base.cc:1034] running on GCE node
I20260925 05:15:57.995055 23864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:57.995280 23864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:57.996431 23864 hybrid_clock.cc:648] HybridClock initialized: now 1790313357996409 us; error 35 us; skew 500 ppm
I20260925 05:15:57.997766 23864 webserver.cc:460] Webserver started at http://127.13.160.130:43051/ using document root <none> and password file <none>
I20260925 05:15:57.997967 23864 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:57.998008 23864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:57.998098 23864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:57.999244 23864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "bc955dfb29eb4fd1a921e8b23d102a90"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "dcab52dcfc9c062548de3cc29b71efbb"
server_key_iv: "9d039a112fe316a6d796f2ecc565f8c9"
server_key_version: "encryptionkey@0"
I20260925 05:15:57.999554 23864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "bc955dfb29eb4fd1a921e8b23d102a90"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "dcab52dcfc9c062548de3cc29b71efbb"
server_key_iv: "9d039a112fe316a6d796f2ecc565f8c9"
server_key_version: "encryptionkey@0"
I20260925 05:15:58.000831 23864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:58.001583 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.001778 23864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:15:58.001871 23864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "bc955dfb29eb4fd1a921e8b23d102a90"
format_stamp: "Formatted at 2026-09-25 05:15:57 on dist-test-slave-9tgz"
server_key: "dcab52dcfc9c062548de3cc29b71efbb"
server_key_iv: "9d039a112fe316a6d796f2ecc565f8c9"
server_key_version: "encryptionkey@0"
I20260925 05:15:58.001945 23864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:58.005405 23864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:58.005688 23864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:58.005827 23864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:58.006039 23864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:58.006371 23864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:58.006428 23864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.006469 23864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:58.006496 23864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.012343 23864 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:40103
I20260925 05:15:58.012421 23991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:40103 every 8 connection(s)
I20260925 05:15:58.012765 23864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:58.015928 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:58.013498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:49267 (local address 127.13.160.190:36545)
0925 05:15:58.013545 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:58.013548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:58.013800 (+ 252us) server_negotiation.cc:407] Connection header received
0925 05:15:58.014335 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:58.014337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:58.014343 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:58.014402 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:58.015650 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:58.015652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:58.015653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:58.015658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:58.015673 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:58.015678 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:58.015732 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:58.015835 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 05:15:58.015852 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:15:58.016137 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:58.013469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36545 (local address 127.13.160.130:49267)
0925 05:15:58.013645 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:58.013709 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:58.014219 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:58.014478 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:58.014482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:58.014494 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:58.015480 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:58.015480 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:58.015510 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:58.015539 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:58.015736 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:58.015741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:58.015745 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:58.015848 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 05:15:58.015913 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260925 05:15:58.016610 23992 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36545
I20260925 05:15:58.016690 23992 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:58.016868 23992 heartbeater.cc:511] Master 127.13.160.190:36545 requested a full tablet report, sending...
I20260925 05:15:58.017254 23673 ts_manager.cc:194] Registered new tserver with Master: bc955dfb29eb4fd1a921e8b23d102a90 (127.13.160.130:40103)
I20260925 05:15:58.017434 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23864
I20260925 05:15:58.017542 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:58.017740 13954 external_mini_cluster.cc:1428] Setting key f68178f6d6b62c0f62f416e8b15bc591
I20260925 05:15:58.018589 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--builtin_ntp_servers=127.13.160.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 05:15:58.101632 23995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:58.101864 23995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:58.101888 23995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:58.103124 23995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:58.103171 23995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:58.103199 23995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:58.104631 23995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:58.104843 23995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:58.105103 23995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:58.110184 24001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:58.110208 24000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:58.110193 24003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:58.110427 23995 server_base.cc:1034] running on GCE node
I20260925 05:15:58.110682 23995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:58.110993 23995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:58.112169 23995 hybrid_clock.cc:648] HybridClock initialized: now 1790313358112153 us; error 47 us; skew 500 ppm
I20260925 05:15:58.113585 23995 webserver.cc:460] Webserver started at http://127.13.160.131:45445/ using document root <none> and password file <none>
I20260925 05:15:58.113986 23995 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:58.114089 23995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:58.114269 23995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:58.115520 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "000956bafc054e3596b7f846717bd1c6"
format_stamp: "Formatted at 2026-09-25 05:15:58 on dist-test-slave-9tgz"
server_key: "903d4b7b588f40eeb474d84211fd7396"
server_key_iv: "380f169b5fa4ac38b5e114dbc29325df"
server_key_version: "encryptionkey@0"
I20260925 05:15:58.115924 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "000956bafc054e3596b7f846717bd1c6"
format_stamp: "Formatted at 2026-09-25 05:15:58 on dist-test-slave-9tgz"
server_key: "903d4b7b588f40eeb474d84211fd7396"
server_key_iv: "380f169b5fa4ac38b5e114dbc29325df"
server_key_version: "encryptionkey@0"
I20260925 05:15:58.117370 23995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:15:58.118186 24009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.118359 23995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:15:58.118467 23995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "000956bafc054e3596b7f846717bd1c6"
format_stamp: "Formatted at 2026-09-25 05:15:58 on dist-test-slave-9tgz"
server_key: "903d4b7b588f40eeb474d84211fd7396"
server_key_iv: "380f169b5fa4ac38b5e114dbc29325df"
server_key_version: "encryptionkey@0"
I20260925 05:15:58.118546 23995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:58.122008 23995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:58.122279 23995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:58.122423 23995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:58.122634 23995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:58.123003 23995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:58.123054 23995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.123093 23995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:58.123121 23995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:58.128928 23995 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:45711
I20260925 05:15:58.128968 24122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:45711 every 8 connection(s)
I20260925 05:15:58.129290 23995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:15:58.132320 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:58.129961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:34937 (local address 127.13.160.190:36545)
0925 05:15:58.130030 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:58.130033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:58.130331 (+ 298us) server_negotiation.cc:407] Connection header received
0925 05:15:58.130799 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:58.130800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:58.130806 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:58.130861 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:58.132129 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:58.132130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:58.132131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:15:58.132134 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:58.132145 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:15:58.132148 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:58.132193 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:58.132255 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 05:15:58.132266 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:15:58.132548 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:58.129963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36545 (local address 127.13.160.131:34937)
0925 05:15:58.130171 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:58.130232 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:58.130713 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:58.130976 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:58.130979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:58.130988 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:58.131983 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:15:58.131984 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:58.132014 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:15:58.132043 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:58.132193 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:58.132196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:58.132198 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:58.132250 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 05:15:58.132300 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260925 05:15:58.133033 24123 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36545
I20260925 05:15:58.133136 24123 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:58.133312 24123 heartbeater.cc:511] Master 127.13.160.190:36545 requested a full tablet report, sending...
I20260925 05:15:58.133458 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 23995
I20260925 05:15:58.133551 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:58.133728 23673 ts_manager.cc:194] Registered new tserver with Master: 000956bafc054e3596b7f846717bd1c6 (127.13.160.131:45711)
I20260925 05:15:58.133760 13954 external_mini_cluster.cc:1428] Setting key ba17615172a56ac49e5ef2683bd759bc
I20260925 05:15:58.134544 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 05:15:58.136523 23713 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:15:58.136032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.13.160.190:36545)
0925 05:15:58.136070 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:58.136072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:58.136202 (+ 130us) server_negotiation.cc:407] Connection header received
0925 05:15:58.136271 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:58.136272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:58.136277 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:58.136326 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:58.136441 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53534: BlockingRecv error: recv got EOF from 127.0.0.1:53534 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:15:58.136637 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23733
I20260925 05:15:58.141580 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23864
I20260925 05:15:58.145710 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23995
I20260925 05:15:58.149966 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 23642
2026-09-25T05:15:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (545 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:15:58 dist-test-slave-9tgz krb5kdc[24133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:15:58 dist-test-slave-9tgz krb5kdc[24133](info): set up 2 sockets
Sep 25 05:15:58 dist-test-slave-9tgz krb5kdc[24133](info): commencing operation
krb5kdc: starting...
W20260925 05:15:59.192634 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:15:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:15:59.313704 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38083
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:38083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:59.397680 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:59.397907 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:59.397930 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:59.399094 24146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:15:59.399139 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:59.399153 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:59.399166 24146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:15:59.399178 24146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:15:59.400605 24146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:38083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:59.400810 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:59.400988 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:59.406019 24152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.406038 24151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.406041 24146 server_base.cc:1034] running on GCE node
W20260925 05:15:59.406026 24154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.406481 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:59.406778 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:59.407919 24146 hybrid_clock.cc:648] HybridClock initialized: now 1790313359407904 us; error 51 us; skew 500 ppm
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:59.410316 24146 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:15:59.410741 24146 webserver.cc:460] Webserver started at http://127.13.160.190:36731/ using document root <none> and password file <none>
I20260925 05:15:59.411026 24146 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:59.411073 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:59.411222 24146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:59.412295 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "cfeab1eefa1e46d19cb72b90099a6083"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "0d72ceacd5520eb44b410eaa135ac6dd"
server_key_iv: "d928464e93279ccf6cc23a163892f22d"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.412662 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "cfeab1eefa1e46d19cb72b90099a6083"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "0d72ceacd5520eb44b410eaa135ac6dd"
server_key_iv: "d928464e93279ccf6cc23a163892f22d"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.413970 24146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:59.414667 24161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.414839 24146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:59.414963 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "cfeab1eefa1e46d19cb72b90099a6083"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "0d72ceacd5520eb44b410eaa135ac6dd"
server_key_iv: "d928464e93279ccf6cc23a163892f22d"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.415040 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:59.424919 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:59.427366 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:59.427532 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:59.431366 24146 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:38083
I20260925 05:15:59.431416 24213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:38083 every 8 connection(s)
I20260925 05:15:59.431787 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:15:59.432793 24214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:59.435216 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: Bootstrap starting.
I20260925 05:15:59.435811 24214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:59.436116 24214 log.cc:824] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:59.436841 24214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: No bootstrap required, opened a new log
I20260925 05:15:59.438484 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 24146
I20260925 05:15:59.438421 24214 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } }
I20260925 05:15:59.438562 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:15:59.438576 24214 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:59.438625 24214 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfeab1eefa1e46d19cb72b90099a6083, State: Initialized, Role: FOLLOWER
I20260925 05:15:59.438742 13954 external_mini_cluster.cc:1428] Setting key 2758e486ff78249e616b24803970ecf7
I20260925 05:15:59.438820 24214 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } }
I20260925 05:15:59.438954 24214 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:15:59.439107 24214 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:15:59.439169 24214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:59.439841 24214 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } }
I20260925 05:15:59.439963 24214 leader_election.cc:302] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfeab1eefa1e46d19cb72b90099a6083; no voters:
I20260925 05:15:59.440148 24214 leader_election.cc:288] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:15:59.440177 24219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:59.440335 24219 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [term 1 LEADER]: Becoming Leader. State: Replica: cfeab1eefa1e46d19cb72b90099a6083, State: Running, Role: LEADER
I20260925 05:15:59.440419 24214 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:15:59.440426 24219 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } }
I20260925 05:15:59.440991 24220 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfeab1eefa1e46d19cb72b90099a6083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } } }
I20260925 05:15:59.441224 24220 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:59.441147 24221 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfeab1eefa1e46d19cb72b90099a6083. Latest consensus state: current_term: 1 leader_uuid: "cfeab1eefa1e46d19cb72b90099a6083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfeab1eefa1e46d19cb72b90099a6083" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38083 } } }
I20260925 05:15:59.441474 24221 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083 [sys.catalog]: This master's current role is: LEADER
I20260925 05:15:59.441802 24228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:15:59.442394 24228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:15:59.443949 24228 catalog_manager.cc:1347] Generated new cluster ID: 4c2bd8bb780f42a5bcb2596458497fa7
I20260925 05:15:59.444008 24228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:59.450652 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.439289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.13.160.190:38083)
0925 05:15:59.439470 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.439480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.439517 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:15:59.439841 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.439846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.439889 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.440042 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:59.440875 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.442349 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.443425 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.443956 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.444062 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.446870 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:59.446874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:59.446875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:59.446909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:59.448943 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.449333 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.449337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.449337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.449503 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.449711 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.449712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.449712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.450035 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:59.450209 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.450284 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.450429 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260925 05:15:59.464620 24228 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:15:59.465204 24228 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:15:59.471952 24228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: Generated new TSK 0
I20260925 05:15:59.472159 24228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:15:59.501969 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:59.583864 24240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:59.584071 24240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:59.584095 24240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:59.585172 24240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:59.585208 24240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:59.585234 24240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:15:59.586715 24240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:59.586967 24240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:59.587165 24240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:59.591992 24246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.592008 24245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.592006 24248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.592180 24240 server_base.cc:1034] running on GCE node
I20260925 05:15:59.592450 24240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:59.592657 24240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:59.593858 24240 hybrid_clock.cc:648] HybridClock initialized: now 1790313359593836 us; error 37 us; skew 500 ppm
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:59.596419 24240 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:15:59.596905 24240 webserver.cc:460] Webserver started at http://127.13.160.129:33797/ using document root <none> and password file <none>
I20260925 05:15:59.597139 24240 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:59.597206 24240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:59.597362 24240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:59.598397 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "0a013bb346de4c9ea217ac767695fb9c"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "8d38d1f39e8645a8cea69fd7fe15f833"
server_key_iv: "ca7bebbd623b36a38c70e103296ab861"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.598752 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "0a013bb346de4c9ea217ac767695fb9c"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "8d38d1f39e8645a8cea69fd7fe15f833"
server_key_iv: "ca7bebbd623b36a38c70e103296ab861"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.600071 24240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:15:59.600797 24255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.600967 24240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:59.601065 24240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "0a013bb346de4c9ea217ac767695fb9c"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "8d38d1f39e8645a8cea69fd7fe15f833"
server_key_iv: "ca7bebbd623b36a38c70e103296ab861"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.601148 24240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:59.609858 24240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:59.612191 24240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:59.612360 24240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:59.612586 24240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:59.612890 24240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:59.612941 24240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.612982 24240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:59.613008 24240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.618970 24240 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:37917
I20260925 05:15:59.619028 24368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:37917 every 8 connection(s)
I20260925 05:15:59.619366 24240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:59.626749 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 24240
I20260925 05:15:59.626863 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:15:59.627084 13954 external_mini_cluster.cc:1428] Setting key a712fbd9b4ac6f82e48cb5fdd43fd219
I20260925 05:15:59.628753 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.620070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:42989 (local address 127.13.160.190:38083)
0925 05:15:59.620123 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.620126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.620384 (+ 258us) server_negotiation.cc:407] Connection header received
0925 05:15:59.620944 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.620945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.620959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.621010 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:59.621906 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.622407 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.623443 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.623632 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.623699 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.625816 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:59.625818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:59.625818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:59.625825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:59.627256 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.627643 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.627644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.627644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.627712 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.628085 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.628085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.628086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.628194 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:59.628258 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.628593 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.628663 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:15:59.629097 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.620066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38083 (local address 127.13.160.129:42989)
0925 05:15:59.620235 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.620270 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.620852 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.621065 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.621068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.621624 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:59.621827 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.621832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.622487 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.622487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.623372 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.623375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.623717 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.623718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.623820 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.624388 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:59.624396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:59.625728 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:59.627318 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.627325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.627327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.627559 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.627827 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.627829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.627829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.628009 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.628274 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:59.628281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:59.628456 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.628571 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.628748 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260925 05:15:59.629673 24369 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38083
I20260925 05:15:59.629772 24369 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:59.629974 24369 heartbeater.cc:511] Master 127.13.160.190:38083 requested a full tablet report, sending...
I20260925 05:15:59.630548 24178 ts_manager.cc:194] Registered new tserver with Master: 0a013bb346de4c9ea217ac767695fb9c (127.13.160.129:37917)
I20260925 05:15:59.631260 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:42989
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:15:59.653493 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:59.734285 24374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:59.734486 24374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:59.734510 24374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:59.735591 24374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:59.735630 24374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:59.735656 24374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:15:59.737031 24374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:59.737228 24374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:59.737394 24374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:59.742447 24379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.742439 24380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.742434 24382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.742704 24374 server_base.cc:1034] running on GCE node
I20260925 05:15:59.742884 24374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:59.743147 24374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:59.744279 24374 hybrid_clock.cc:648] HybridClock initialized: now 1790313359744268 us; error 27 us; skew 500 ppm
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:59.747182 24374 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:15:59.747664 24374 webserver.cc:460] Webserver started at http://127.13.160.130:45885/ using document root <none> and password file <none>
I20260925 05:15:59.747946 24374 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:59.748023 24374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:59.748157 24374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:59.749197 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "db8919718e6e4caf99a2e7e745f6144a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "81104f60af11dbeafa3b6154a0a49e9b"
server_key_iv: "99431478ac5040cfc34e2eaef8ca95e5"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.749562 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "db8919718e6e4caf99a2e7e745f6144a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "81104f60af11dbeafa3b6154a0a49e9b"
server_key_iv: "99431478ac5040cfc34e2eaef8ca95e5"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.750874 24374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 05:15:59.751670 24389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.751845 24374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:15:59.751942 24374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "db8919718e6e4caf99a2e7e745f6144a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "81104f60af11dbeafa3b6154a0a49e9b"
server_key_iv: "99431478ac5040cfc34e2eaef8ca95e5"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.752029 24374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:59.758441 24374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:59.760779 24374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:59.760959 24374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:59.761190 24374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:59.761492 24374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:59.761544 24374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.761585 24374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:59.761613 24374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.767457 24374 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38585
I20260925 05:15:59.767496 24502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38585 every 8 connection(s)
I20260925 05:15:59.767804 24374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:15:59.768328 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 24374
I20260925 05:15:59.768435 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:15:59.768625 13954 external_mini_cluster.cc:1428] Setting key ab3a654a853bf1c0d0114b7e8a8eb4b1
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
I20260925 05:15:59.777513 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.768482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:60173 (local address 127.13.160.190:38083)
0925 05:15:59.768532 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.768534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.768767 (+ 233us) server_negotiation.cc:407] Connection header received
0925 05:15:59.769434 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.769435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.769448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.769504 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:59.770453 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.771047 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.772031 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.772246 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.772324 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.774695 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:59.774696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:59.774697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:59.774704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:59.776120 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.776566 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.776567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.776567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.776625 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.776881 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.776882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.776882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.776995 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:59.777073 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.777343 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.777420 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:15:59.777709 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.768478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38083 (local address 127.13.160.130:60173)
0925 05:15:59.768632 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.768688 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.769331 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.769574 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.769578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.770150 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:59.770344 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.770349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.771147 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.771148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.771936 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.771939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.772330 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.772332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.772459 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.772997 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:59.773005 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:59.774591 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:59.776231 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.776238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.776240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.776478 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.776705 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.776706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.776706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.776808 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.777063 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:59.777069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:59.777213 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.777322 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.777473 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260925 05:15:59.778234 24503 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38083
I20260925 05:15:59.778337 24503 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:59.778512 24503 heartbeater.cc:511] Master 127.13.160.190:38083 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
I20260925 05:15:59.778956 24178 ts_manager.cc:194] Registered new tserver with Master: db8919718e6e4caf99a2e7e745f6144a (127.13.160.130:38585)
I20260925 05:15:59.779331 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:60173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:15:59.791570 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:15:59.871347 24508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:15:59.871557 24508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:15:59.871582 24508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:15:59.872654 24508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:15:59.872687 24508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:15:59.872713 24508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:15:59.874054 24508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:15:59.874238 24508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:15:59.874406 24508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:15:59.879298 24514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:15:59.879298 24513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.879549 24508 server_base.cc:1034] running on GCE node
W20260925 05:15:59.879619 24516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:15:59.879890 24508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:15:59.880177 24508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:15:59.881347 24508 hybrid_clock.cc:648] HybridClock initialized: now 1790313359881345 us; error 37 us; skew 500 ppm
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:15:59.883922 24508 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:15:59.884346 24508 webserver.cc:460] Webserver started at http://127.13.160.131:36999/ using document root <none> and password file <none>
I20260925 05:15:59.884569 24508 fs_manager.cc:359] Metadata directory not provided
I20260925 05:15:59.884639 24508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:15:59.884795 24508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:15:59.885875 24508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "5ff383eb52c54f34aabf3312b74b5b0a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "cc1e8eeedf7a12b99960f0d1b6889c47"
server_key_iv: "c2dae580b6812ad133b108d18460d3d3"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.886240 24508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "5ff383eb52c54f34aabf3312b74b5b0a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "cc1e8eeedf7a12b99960f0d1b6889c47"
server_key_iv: "c2dae580b6812ad133b108d18460d3d3"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.887624 24508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:15:59.888320 24523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.888484 24508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.888588 24508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "5ff383eb52c54f34aabf3312b74b5b0a"
format_stamp: "Formatted at 2026-09-25 05:15:59 on dist-test-slave-9tgz"
server_key: "cc1e8eeedf7a12b99960f0d1b6889c47"
server_key_iv: "c2dae580b6812ad133b108d18460d3d3"
server_key_version: "encryptionkey@0"
I20260925 05:15:59.888684 24508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:15:59.899677 24508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:15:59.901938 24508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:15:59.902099 24508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:15:59.902331 24508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:15:59.902627 24508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:15:59.902688 24508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.902734 24508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:15:59.902762 24508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:15:59.908870 24508 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:46279
I20260925 05:15:59.908907 24636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:46279 every 8 connection(s)
I20260925 05:15:59.909238 24508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:15:59 dist-test-slave-9tgz krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313359, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:15:59.916030 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 24508
I20260925 05:15:59.916141 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:15:59.916381 13954 external_mini_cluster.cc:1428] Setting key e634a4c4f5503893b34adafb9ca2b66d
I20260925 05:15:59.918715 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.910063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:38969 (local address 127.13.160.190:38083)
0925 05:15:59.910135 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.910137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.910322 (+ 185us) server_negotiation.cc:407] Connection header received
0925 05:15:59.910797 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.910798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.910808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.910869 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:59.911862 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.912368 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.913387 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.913571 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.913625 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.915877 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:59.915879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:59.915879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:59.915887 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:59.917559 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.917924 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.917925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.917925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.917975 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.918147 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.918147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.918147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.918265 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:59.918311 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.918590 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.918650 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 05:15:59.918998 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.909995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38083 (local address 127.13.160.131:38969)
0925 05:15:59.910186 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.910228 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.910712 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.910982 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.910985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.911550 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:15:59.911752 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.911757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.912460 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.912461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.913303 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.913305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.913660 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.913660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.913759 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.914335 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:15:59.914343 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:15:59.915779 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:15:59.917624 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.917629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.917631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.917860 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.918015 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:15:59.918016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:15:59.918016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:15:59.918097 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:15:59.918313 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:15:59.918318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:15:59.918477 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.918570 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.918703 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260925 05:15:59.919479 24637 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38083
I20260925 05:15:59.919564 24637 heartbeater.cc:464] Registering TS with master...
I20260925 05:15:59.919754 24637 heartbeater.cc:511] Master 127.13.160.190:38083 requested a full tablet report, sending...
I20260925 05:15:59.920118 24178 ts_manager.cc:194] Registered new tserver with Master: 5ff383eb52c54f34aabf3312b74b5b0a (127.13.160.131:46279)
I20260925 05:15:59.920452 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:38969
I20260925 05:15:59.928215 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:15:59.935256 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.929506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.13.160.190:38083)
0925 05:15:59.929582 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.929585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.929678 (+ 93us) server_negotiation.cc:407] Connection header received
0925 05:15:59.929733 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.929733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.929743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.929780 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:15:59.930296 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.930753 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.931485 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.931662 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.931714 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.932815 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:15:59.932817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:15:59.932817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:15:59.932822 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:15:59.934217 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.934495 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.934496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.934496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.934544 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:15:59.934689 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:15:59.934690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:15:59.934690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:15:59.934813 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:15:59.934880 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.935064 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.935141 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:15:59.936715 24178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:15:59.938251 24178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:15:59.944661 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:46279 (local address 127.0.0.1:41304)
0925 05:15:59.941024 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.941033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.941083 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.941433 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.941436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.941438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:59.941505 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.941508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.942555 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.942556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.943625 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.943627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.944370 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.944442 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.944456 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.944526 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.944574 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":51,"threads_started":1}
I20260925 05:15:59.944677 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.13.160.131:46279)
0925 05:15:59.940988 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.940991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.941164 (+ 173us) server_negotiation.cc:407] Connection header received
0925 05:15:59.941212 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.941214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.941232 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.941322 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:59.941589 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.942456 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.943725 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.944293 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944372 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.944423 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.944554 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.944594 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
I20260925 05:15:59.945194 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37917 (local address 127.0.0.1:59982)
0925 05:15:59.940956 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.940999 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.941083 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.941448 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.941448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.941451 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:59.941516 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.941517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.942863 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.942864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.944018 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.944020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.944923 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.944996 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.945000 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.945078 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.945119 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":66,"threads_started":1}
I20260925 05:15:59.945220 24648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.13.160.129:37917)
0925 05:15:59.940952 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.940957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.941063 (+ 106us) server_negotiation.cc:407] Connection header received
0925 05:15:59.941347 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.941349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.941368 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.941451 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:59.941604 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.942753 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944107 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.944837 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944916 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.944967 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.945108 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.945144 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260925 05:15:59.945446 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38585 (local address 127.0.0.1:36536)
0925 05:15:59.941267 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.941276 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.941321 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.941880 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.941881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.941883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:59.941921 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.941922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.943167 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.943168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.944215 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.944217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.945201 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.945202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.945272 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.945276 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.945327 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.945384 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260925 05:15:59.945739 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.940782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.13.160.130:38585)
0925 05:15:59.940981 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.940986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.941586 (+ 600us) server_negotiation.cc:407] Connection header received
0925 05:15:59.941631 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.941633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.941660 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.941731 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:59.942021 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.943069 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.944293 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.944998 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.945102 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.945170 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.945607 (+ 437us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.945654 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
I20260925 05:15:59.945737 24571 tablet_service.cc:1467] Processing CreateTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c (DEFAULT_TABLE table=test-table [id=a5b741de4e6546da8d9f529dc8925cb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:59.945823 24303 tablet_service.cc:1467] Processing CreateTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c (DEFAULT_TABLE table=test-table [id=a5b741de4e6546da8d9f529dc8925cb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:59.946079 24303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92cd152ad406487a9fa3f0e81b988d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:59.946084 24571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92cd152ad406487a9fa3f0e81b988d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:59.946280 24437 tablet_service.cc:1467] Processing CreateTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c (DEFAULT_TABLE table=test-table [id=a5b741de4e6546da8d9f529dc8925cb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:15:59.946537 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92cd152ad406487a9fa3f0e81b988d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:15:59.948731 24651 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: Bootstrap starting.
I20260925 05:15:59.949298 24652 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Bootstrap starting.
I20260925 05:15:59.949364 24651 tablet_bootstrap.cc:654] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:59.949966 24652 tablet_bootstrap.cc:654] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:59.949957 24651 log.cc:824] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:59.950029 24653 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Bootstrap starting.
I20260925 05:15:59.950486 24652 log.cc:824] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:59.950661 24653 tablet_bootstrap.cc:654] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Neither blocks nor log segments found. Creating new log.
I20260925 05:15:59.950737 24651 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: No bootstrap required, opened a new log
I20260925 05:15:59.950829 24651 ts_tablet_manager.cc:1397] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:59.951187 24653 log.cc:824] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:15:59.951400 24652 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: No bootstrap required, opened a new log
I20260925 05:15:59.951469 24652 ts_tablet_manager.cc:1397] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:59.951843 24653 tablet_bootstrap.cc:492] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: No bootstrap required, opened a new log
I20260925 05:15:59.951911 24653 ts_tablet_manager.cc:1397] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:15:59.953011 24651 raft_consensus.cc:348] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953199 24651 raft_consensus.cc:374] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:59.953238 24651 raft_consensus.cc:729] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a013bb346de4c9ea217ac767695fb9c, State: Initialized, Role: FOLLOWER
I20260925 05:15:59.953176 24653 raft_consensus.cc:348] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953365 24653 raft_consensus.cc:374] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:59.953409 24653 raft_consensus.cc:729] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8919718e6e4caf99a2e7e745f6144a, State: Initialized, Role: FOLLOWER
I20260925 05:15:59.953392 24651 consensus_queue.cc:260] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953347 24652 raft_consensus.cc:348] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953474 24652 raft_consensus.cc:374] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:15:59.953521 24652 raft_consensus.cc:729] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff383eb52c54f34aabf3312b74b5b0a, State: Initialized, Role: FOLLOWER
I20260925 05:15:59.953553 24653 consensus_queue.cc:260] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953657 24651 ts_tablet_manager.cc:1428] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 05:15:59.953657 24652 consensus_queue.cc:260] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.953776 24369 heartbeater.cc:503] Master 127.13.160.190:38083 was elected leader, sending a full tablet report...
I20260925 05:15:59.953836 24653 ts_tablet_manager.cc:1428] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:59.953912 24652 ts_tablet_manager.cc:1428] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:15:59.953948 24503 heartbeater.cc:503] Master 127.13.160.190:38083 was elected leader, sending a full tablet report...
I20260925 05:15:59.954107 24637 heartbeater.cc:503] Master 127.13.160.190:38083 was elected leader, sending a full tablet report...
I20260925 05:15:59.980067 24658 raft_consensus.cc:482] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:15:59.980183 24658 raft_consensus.cc:504] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.980532 24658 leader_election.cc:288] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a013bb346de4c9ea217ac767695fb9c (127.13.160.129:37917), 5ff383eb52c54f34aabf3312b74b5b0a (127.13.160.131:46279)
I20260925 05:15:59.983399 24648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.980764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:57439 (local address 127.13.160.129:37917)
0925 05:15:59.980808 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.980810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.980823 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:15:59.980908 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.980910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.980919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.980999 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:59.981180 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.981664 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.982583 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.983025 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.983112 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.983120 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.983280 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.983320 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:15:59.983409 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.980813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39933 (local address 127.13.160.131:46279)
0925 05:15:59.980853 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.980855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.980980 (+ 125us) server_negotiation.cc:407] Connection header received
0925 05:15:59.981058 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.981060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:15:59.981070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:15:59.981140 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:15:59.981295 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.981756 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.982657 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.983054 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.983138 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.983145 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:15:59.983304 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 05:15:59.983340 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:15:59.983428 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.980717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37917 (local address 127.13.160.130:57439)
0925 05:15:59.980780 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.980794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.980838 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.981003 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.981003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.981005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:59.981083 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.981084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.981750 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.981751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.982489 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.982490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.983109 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.983110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.983195 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.983197 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.983282 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.983320 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 05:15:59.983438 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.980747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:46279 (local address 127.13.160.130:39933)
0925 05:15:59.980907 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 05:15:59.980915 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:15:59.980991 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:15:59.981143 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:15:59.981144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:15:59.981145 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:15:59.981225 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.981227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.981842 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.981843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.982557 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:15:59.982559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:15:59.983137 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:15:59.983137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:15:59.983217 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:15:59.983220 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:15:59.983295 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 05:15:59.983333 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260925 05:15:59.983811 24323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92cd152ad406487a9fa3f0e81b988d6c" candidate_uuid: "db8919718e6e4caf99a2e7e745f6144a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a013bb346de4c9ea217ac767695fb9c" is_pre_election: true
I20260925 05:15:59.983984 24323 raft_consensus.cc:2454] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8919718e6e4caf99a2e7e745f6144a in term 0.
I20260925 05:15:59.983949 24591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92cd152ad406487a9fa3f0e81b988d6c" candidate_uuid: "db8919718e6e4caf99a2e7e745f6144a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" is_pre_election: true
I20260925 05:15:59.984079 24591 raft_consensus.cc:2454] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8919718e6e4caf99a2e7e745f6144a in term 0.
I20260925 05:15:59.984215 24390 leader_election.cc:302] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a013bb346de4c9ea217ac767695fb9c, db8919718e6e4caf99a2e7e745f6144a; no voters:
I20260925 05:15:59.984388 24658 raft_consensus.cc:2781] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:15:59.984445 24658 raft_consensus.cc:482] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:15:59.984480 24658 raft_consensus.cc:3037] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:59.985204 24658 raft_consensus.cc:504] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.985347 24658 leader_election.cc:288] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [CANDIDATE]: Term 1 election: Requested vote from peers 0a013bb346de4c9ea217ac767695fb9c (127.13.160.129:37917), 5ff383eb52c54f34aabf3312b74b5b0a (127.13.160.131:46279)
I20260925 05:15:59.985471 24323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92cd152ad406487a9fa3f0e81b988d6c" candidate_uuid: "db8919718e6e4caf99a2e7e745f6144a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a013bb346de4c9ea217ac767695fb9c"
I20260925 05:15:59.985530 24591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92cd152ad406487a9fa3f0e81b988d6c" candidate_uuid: "db8919718e6e4caf99a2e7e745f6144a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff383eb52c54f34aabf3312b74b5b0a"
I20260925 05:15:59.985549 24323 raft_consensus.cc:3037] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:59.985595 24591 raft_consensus.cc:3037] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:15:59.986438 24323 raft_consensus.cc:2454] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8919718e6e4caf99a2e7e745f6144a in term 1.
I20260925 05:15:59.986438 24591 raft_consensus.cc:2454] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8919718e6e4caf99a2e7e745f6144a in term 1.
I20260925 05:15:59.986593 24390 leader_election.cc:302] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a013bb346de4c9ea217ac767695fb9c, db8919718e6e4caf99a2e7e745f6144a; no voters:
I20260925 05:15:59.986675 24658 raft_consensus.cc:2781] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:15:59.986840 24658 raft_consensus.cc:686] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 1 LEADER]: Becoming Leader. State: Replica: db8919718e6e4caf99a2e7e745f6144a, State: Running, Role: LEADER
I20260925 05:15:59.986959 24658 consensus_queue.cc:237] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } }
I20260925 05:15:59.987790 24176 catalog_manager.cc:5495] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a reported cstate change: term changed from 0 to 1, leader changed from <none> to db8919718e6e4caf99a2e7e745f6144a (127.13.160.130). New cstate: current_term: 1 leader_uuid: "db8919718e6e4caf99a2e7e745f6144a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8919718e6e4caf99a2e7e745f6144a" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 } health_report { overall_health: UNKNOWN } } }
I20260925 05:16:00.002288 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:15:59.999871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.13.160.130:38585)
0925 05:15:59.999933 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:15:59.999935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:15:59.999944 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:15:59.999992 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:15:59.999993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:00.000009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:00.000088 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:16:00.000255 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:00.000726 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:00.001472 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:00.001663 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:00.001770 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:00.001936 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:16:00.002043 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:16:00.002137 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:00.002156 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 05:16:00.002174 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:16:00.004338 24591 raft_consensus.cc:1261] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 1 FOLLOWER]: Refusing update from remote peer db8919718e6e4caf99a2e7e745f6144a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:16:00.004323 24323 raft_consensus.cc:1261] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 1 FOLLOWER]: Refusing update from remote peer db8919718e6e4caf99a2e7e745f6144a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:16:00.004747 24658 consensus_queue.cc:1035] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a013bb346de4c9ea217ac767695fb9c" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:16:00.004921 24661 consensus_queue.cc:1035] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff383eb52c54f34aabf3312b74b5b0a" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 46279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:16:00.007380 24665 mvcc.cc:205] Tried to move back new op lower bound from 7333123522574495744 to 7333123522507309056. Current Snapshot: MvccSnapshot[applied={T|T < 7333123522574495744}]
I20260925 05:16:00.007175 24664 mvcc.cc:205] Tried to move back new op lower bound from 7333123522574495744 to 7333123522507309056. Current Snapshot: MvccSnapshot[applied={T|T < 7333123522574495744}]
I20260925 05:16:00.007700 24666 mvcc.cc:205] Tried to move back new op lower bound from 7333123522574495744 to 7333123522507309056. Current Snapshot: MvccSnapshot[applied={T|T < 7333123522574495744}]
I20260925 05:16:00.010040 24176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:16:00.010140 24176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:16:00.010906 24176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfeab1eefa1e46d19cb72b90099a6083: Sending DeleteTablet for 3 replicas of tablet 92cd152ad406487a9fa3f0e81b988d6c
I20260925 05:16:00.011232 24437 tablet_service.cc:1514] Processing DeleteTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:36536
I20260925 05:16:00.011251 24571 tablet_service.cc:1514] Processing DeleteTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260925 05:16:00.011468 24674 tablet_replica.cc:330] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: stopping tablet replica
I20260925 05:16:00.011462 24303 tablet_service.cc:1514] Processing DeleteTablet for tablet 92cd152ad406487a9fa3f0e81b988d6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:59982
I20260925 05:16:00.011564 24674 raft_consensus.cc:2229] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:16:00.011682 24676 tablet_replica.cc:330] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c: stopping tablet replica
I20260925 05:16:00.011684 24674 raft_consensus.cc:2258] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:16:00.011770 24676 raft_consensus.cc:2229] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:16:00.011868 24676 raft_consensus.cc:2258] T 92cd152ad406487a9fa3f0e81b988d6c P 0a013bb346de4c9ea217ac767695fb9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:16:00.012001 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 24240
I20260925 05:16:00.012744 24674 ts_tablet_manager.cc:1905] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:16:00.014415 24675 tablet_replica.cc:330] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: stopping tablet replica
I20260925 05:16:00.014521 24675 raft_consensus.cc:2229] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:16:00.014617 24675 raft_consensus.cc:2258] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:16:00.014914 24674 ts_tablet_manager.cc:1918] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:16:00.014978 24674 log.cc:1196] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/92cd152ad406487a9fa3f0e81b988d6c
I20260925 05:16:00.015255 24674 ts_tablet_manager.cc:1939] T 92cd152ad406487a9fa3f0e81b988d6c P db8919718e6e4caf99a2e7e745f6144a: Deleting consensus metadata
I20260925 05:16:00.015659 24164 catalog_manager.cc:4841] TS db8919718e6e4caf99a2e7e745f6144a (127.13.160.130:38585): tablet 92cd152ad406487a9fa3f0e81b988d6c (table test-table [id=a5b741de4e6546da8d9f529dc8925cb7]) successfully deleted
I20260925 05:16:00.015986 24675 ts_tablet_manager.cc:1905] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:16:00.017588 24675 ts_tablet_manager.cc:1918] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:16:00.017673 24675 log.cc:1196] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/92cd152ad406487a9fa3f0e81b988d6c
I20260925 05:16:00.017993 24675 ts_tablet_manager.cc:1939] T 92cd152ad406487a9fa3f0e81b988d6c P 5ff383eb52c54f34aabf3312b74b5b0a: Deleting consensus metadata
I20260925 05:16:00.018607 24163 catalog_manager.cc:4841] TS 5ff383eb52c54f34aabf3312b74b5b0a (127.13.160.131:46279): tablet 92cd152ad406487a9fa3f0e81b988d6c (table test-table [id=a5b741de4e6546da8d9f529dc8925cb7]) successfully deleted
W20260925 05:16:00.018996 24162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:37917 (error 108)
I20260925 05:16:00.019261 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 24374
I20260925 05:16:00.019441 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:00.019323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37917 (local address 127.0.0.1:59998)
0925 05:16:00.019372 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:00.019404 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:37917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 05:16:00.020159 24162 catalog_manager.cc:4584] TS 0a013bb346de4c9ea217ac767695fb9c (127.13.160.129:37917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92cd152ad406487a9fa3f0e81b988d6c: Network error: Client connection negotiation failed: client connection to 127.13.160.129:37917: connect: Connection refused (error 111)
I20260925 05:16:00.024274 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 24508
I20260925 05:16:00.028853 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 24146
2026-09-25T05:16:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1879 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:16:00 dist-test-slave-9tgz krb5kdc[24680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:16:00 dist-test-slave-9tgz krb5kdc[24680](info): set up 2 sockets
Sep 25 05:16:00 dist-test-slave-9tgz krb5kdc[24680](info): commencing operation
krb5kdc: starting...
W20260925 05:16:01.061061 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:16:01 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:16:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:16:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangeradmin_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangeradmin_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangerlookup_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangerlookup_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
I20260925 05:16:01.227849 13954 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq3iYZX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/postgres -l logfile start
2026-09-25 05:16:03.706 UTC [24710] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 05:16:03.706 UTC [24710] LOG: listening on IPv4 address "127.13.160.148", port 50195
2026-09-25 05:16:03.710 UTC [24710] LOG: listening on Unix socket "/tmp/.s.PGSQL.50195"
2026-09-25 05:16:03.745 UTC [24713] LOG: database system was shut down at 2026-09-25 05:16:02 UTC
2026-09-25 05:16:03.749 UTC [24710] LOG: database system is ready to accept connections
I20260925 05:16:04.688555 13954 mini_postgres.cc:96] Postgres bound to 50195
2026-09-25 05:16:04.694 UTC [24721] FATAL: database "slave" does not exist
127.13.160.148:50195 - accepting connections
I20260925 05:16:04.695467 13954 mini_ranger.cc:161] Starting Ranger...
I20260925 05:16:04.710925 13954 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 05:16:05.174661 13954 mini_ranger.cc:87] Created ranger Postgres database
I20260925 05:16:05.174754 13954 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin
2026-09-25 05:16:05,460 [I] DB FLAVOR :POSTGRES
2026-09-25 05:16:05,461 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 05:16:05,461 [I] Checking connection..
2026-09-25 05:16:05,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 05:16:05,720 [I] Checking connection passed.
2026-09-25 05:16:05,720 [I] --------- Verifying version history table ---------
2026-09-25 05:16:05,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:05,951 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 05:16:05,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:06,213 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 05:16:06,213 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 05:16:06,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 05:16:06.475 UTC [24785] WARNING: there is no transaction in progress
2026-09-25 05:16:06,482 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 05:16:06,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:06,729 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 05:16:06,730 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 05:16:06,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 05:16:06,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 05:16:07,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9tgz.c.gcp-upstream.internal','N') ;"
2026-09-25 05:16:07,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 05:16:07,735 [I] Table x_portal_user does not exist in database ranger
2026-09-25 05:16:07,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 05:16:07,995 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 05:16:07,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 05:16:08,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 05:16:08,480 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 05:16:08,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 05:16:09.466 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.482 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.499 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.515 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.535 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.557 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.626 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.633 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.648 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.659 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.671 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.686 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.697 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.704 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.711 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:09.717 UTC [24893] WARNING: there is no transaction in progress
2026-09-25 05:16:10,375 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 05:16:10,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 05:16:10,615 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 05:16:10,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 05:16:10,855 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 05:16:10,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 05:16:11,105 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 05:16:11,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 05:16:11,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 05:16:11,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9tgz.c.gcp-upstream.internal';"
2026-09-25 05:16:11,839 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 05:16:11,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 05:16:12,102 [I] DB_PATCHES have already been applied
I20260925 05:16:12.108099 13954 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq3iYZX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 05:16:12.108176 13954 mini_ranger.cc:193] Using host: 127.13.160.148
I20260925 05:16:12.111318 13954 mini_ranger.cc:227] Ranger admin URL: http://127.13.160.148:39891
Sep 25, 2026 5:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 5:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 5:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 5:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 5:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 5:16:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 5:16:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 05:16:13.223407 13954 mini_ranger.cc:160] Time spent starting Ranger: real 8.528s user 0.003s sys 0.006s
Sep 25, 2026 5:16:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 5:16:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 5:16:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 5:16:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 5:16:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 05:16:35.315342 13954 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangerkms_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/rangerkms_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.160.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.160.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 05:16:35 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313395, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 05:16:35.409566 13954 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 05:16:35.419368 13954 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 05:16:36.290112 13954 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 05:16:36.290205 13954 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms
I20260925 05:16:36.290227 13954 mini_ranger_kms.cc:227] Using postgres at 127.13.160.148:50195
2026-09-25 05:16:36,483 [I] DB FLAVOR :POSTGRES
2026-09-25 05:16:36,483 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 05:16:36,483 [I] Checking connection
2026-09-25 05:16:36,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 05:16:36,732 [I] connection success
2026-09-25 05:16:36,733 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 05:16:36,733 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 05:16:36,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 05:16:36,993 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 05:16:36,993 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 05:16:36,993 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 05:16:36,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:50195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 05:16:37,255 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 05:16:37.708630 13954 mini_ranger_kms.cc:310] Created kms service
I20260925 05:16:37.931339 13954 mini_ranger_kms.cc:326] Created kudu user
I20260925 05:16:37.973691 13954 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 05:16:37 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313395, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:38.347674 13954 mini_ranger_kms.cc:384] Added ranger policy
I20260925 05:16:38.347805 13954 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 05:16:38.347828 13954 mini_ranger_kms.cc:240] Using host: 127.13.160.148
I20260925 05:16:38.351315 13954 mini_ranger_kms.cc:276] Ranger KMS PID: 25073
I20260925 05:16:38.351459 13954 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.160.148:60595
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 5:16:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 05:16:40 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790313400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:16:40.640728 13954 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.231s user 0.000s sys 0.174s
I20260925 05:16:40.640844 13954 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 05:16:40.640949 13954 mini_ranger_kms.cc:401] 127.13.160.148:60595/kms/v1/keys
Sep 25 05:16:41 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
Sep 25 05:16:41 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
Sep 25 05:16:43 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:16:43.870404 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35237
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:35237
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:60595/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:16:43.979000 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:16:43.979230 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:16:43.979254 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:16:43.980507 25124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:16:43.980561 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:16:43.980575 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:16:43.980588 25124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:16:43.980602 25124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:16:43.982276 25124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.160.148:60595/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:35237
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:35237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:16:43.982554 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:16:43.982837 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:16:43.989248 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:16:43.989253 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:43.989486 25124 server_base.cc:1034] running on GCE node
W20260925 05:16:43.989253 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:43.989820 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:16:43.990197 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:16:43.991353 25124 hybrid_clock.cc:648] HybridClock initialized: now 1790313403991338 us; error 29 us; skew 500 ppm
Sep 25 05:16:43 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313403, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:16:43.994129 25124 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:16:43.994740 25124 webserver.cc:460] Webserver started at http://127.13.160.190:45233/ using document root <none> and password file <none>
I20260925 05:16:43.995075 25124 fs_manager.cc:359] Metadata directory not provided
I20260925 05:16:43.995152 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:16:43.995334 25124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:16:43 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313403, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:44.054164 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "0b351e8bb1e64ce5b1053f9285f1afde"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "3148fcb6bc37e8e06331272b3fea1fac"
server_key_iv: "34dc54f84b127521c66f09f801745747"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:44.054697 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "0b351e8bb1e64ce5b1053f9285f1afde"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "3148fcb6bc37e8e06331272b3fea1fac"
server_key_iv: "34dc54f84b127521c66f09f801745747"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:44.056533 25124 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260925 05:16:44.069134 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:44.069403 25124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:16:44.069504 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "0b351e8bb1e64ce5b1053f9285f1afde"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "3148fcb6bc37e8e06331272b3fea1fac"
server_key_iv: "34dc54f84b127521c66f09f801745747"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:44.069631 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:16:44.093261 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:16:44.095818 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:16:44.095988 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:16:44.101038 25124 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:35237
I20260925 05:16:44.101052 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:35237 every 8 connection(s)
I20260925 05:16:44.101485 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:16:44.102213 25192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:16:44.105201 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: Bootstrap starting.
I20260925 05:16:44.105865 25192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: Neither blocks nor log segments found. Creating new log.
I20260925 05:16:44.106194 25192 log.cc:824] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: Log is configured to *not* fsync() on all Append() calls
I20260925 05:16:44.107007 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: No bootstrap required, opened a new log
I20260925 05:16:44.107827 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 25124
I20260925 05:16:44.107935 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:16:44.108111 13954 external_mini_cluster.cc:1428] Setting key 1b62d69c961dc2ca491b0d0115c03586
I20260925 05:16:44.109668 25192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } }
I20260925 05:16:44.109850 25192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:16:44.109916 25192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b351e8bb1e64ce5b1053f9285f1afde, State: Initialized, Role: FOLLOWER
I20260925 05:16:44.110085 25192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } }
I20260925 05:16:44.110189 25192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:16:44.110224 25192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:16:44.110299 25192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:16:44.111172 25192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } }
I20260925 05:16:44.111303 25192 leader_election.cc:302] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b351e8bb1e64ce5b1053f9285f1afde; no voters:
I20260925 05:16:44.111502 25192 leader_election.cc:288] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:16:44.111531 25197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:16:44.111770 25197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [term 1 LEADER]: Becoming Leader. State: Replica: 0b351e8bb1e64ce5b1053f9285f1afde, State: Running, Role: LEADER
I20260925 05:16:44.111896 25192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:16:44.111896 25197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } }
I20260925 05:16:44.112342 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b351e8bb1e64ce5b1053f9285f1afde. Latest consensus state: current_term: 1 leader_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } } }
I20260925 05:16:44.113449 25192 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 05:16:44.113611 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [sys.catalog]: This master's current role is: LEADER
I20260925 05:16:44.113821 25211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:16:44.113924 25199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b351e8bb1e64ce5b1053f9285f1afde" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 35237 } } }
I20260925 05:16:44.114007 25199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde [sys.catalog]: This master's current role is: LEADER
Sep 25 05:16:44 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:16:44.117754 25211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:16:44.123127 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:44.108684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.13.160.190:35237)
0925 05:16:44.108858 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:44.108868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:44.108899 (+ 31us) server_negotiation.cc:407] Connection header received
0925 05:16:44.109674 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:44.109685 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:44.109764 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:44.109925 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:16:44.110857 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:44.112416 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:44.113579 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:44.114220 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:44.114344 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:44.117682 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:16:44.117734 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:16:44.117735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:16:44.118301 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:16:44.120995 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:44.121455 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:44.121459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:44.121459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:44.121585 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:44.121788 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:44.121789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:44.121790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:44.122222 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:16:44.122385 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:44.122497 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 05:16:44.122805 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260925 05:16:44.127933 25211 catalog_manager.cc:1347] Generated new cluster ID: 7e8cf293045042f8a05557936b8dcddf
I20260925 05:16:44.127998 25211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:16:44.139770 25211 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:16:44.140602 25211 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:16:44.154835 25211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: Generated new TSK 0
I20260925 05:16:44.155238 25211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:16:44.864012 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:60595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:16:44.955168 25237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:16:44.955544 25237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:16:44.955588 25237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:16:44.957034 25237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:16:44.957086 25237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:16:44.957116 25237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:16:44.958716 25237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.160.148:60595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:16:44.958999 25237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:16:44.959295 25237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:16:44.964464 25242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:16:44.964609 25243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:16:44.964780 25245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:44.965893 25237 server_base.cc:1034] running on GCE node
I20260925 05:16:44.966140 25237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:16:44.966450 25237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:16:44.967617 25237 hybrid_clock.cc:648] HybridClock initialized: now 1790313404967594 us; error 39 us; skew 500 ppm
Sep 25 05:16:44 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313404, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:16:44.970650 25237 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:16:44.971312 25237 webserver.cc:460] Webserver started at http://127.13.160.129:42629/ using document root <none> and password file <none>
I20260925 05:16:44.971601 25237 fs_manager.cc:359] Metadata directory not provided
I20260925 05:16:44.971666 25237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:16:44.971817 25237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:16:44 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313404, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:44.985898 25237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "feab14ca93bb4d80abc68f06611a7d38"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "6e724aeba7d96b68b1a46a8d65b9f035"
server_key_iv: "d429206063065d3d507e3d64d9626717"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:44.986474 25237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "feab14ca93bb4d80abc68f06611a7d38"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "6e724aeba7d96b68b1a46a8d65b9f035"
server_key_iv: "d429206063065d3d507e3d64d9626717"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:44.989450 25237 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 05:16:45.002568 25252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.002825 25237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:16:45.002981 25237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "feab14ca93bb4d80abc68f06611a7d38"
format_stamp: "Formatted at 2026-09-25 05:16:44 on dist-test-slave-9tgz"
server_key: "6e724aeba7d96b68b1a46a8d65b9f035"
server_key_iv: "d429206063065d3d507e3d64d9626717"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.003104 25237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:16:45.015818 25237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:16:45.018591 25237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:16:45.018754 25237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:16:45.019151 25237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:16:45.019559 25237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:16:45.019630 25237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.019666 25237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:16:45.019685 25237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.026434 25237 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:41351
I20260925 05:16:45.026867 25237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:16:45.029961 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 25237
I20260925 05:16:45.030076 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:16:45.030313 13954 external_mini_cluster.cc:1428] Setting key 445860c18df341429b8e40a74f93da1f
I20260925 05:16:45.028980 25367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:41351 every 8 connection(s)
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313404, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
I20260925 05:16:45.044106 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.031472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35237 (local address 127.13.160.129:33189)
0925 05:16:45.034045 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.034087 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.034584 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.034988 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.034993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.035206 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:16:45.035420 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.035425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.036236 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.036237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.037070 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.037072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.037526 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.037527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.037689 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.038358 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:16:45.038369 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:16:45.040103 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:16:45.042290 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.042300 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.042303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.042624 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.042864 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.042865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.042865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.043058 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.043375 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:16:45.043381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:16:45.043563 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.043669 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.043852 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2525,"thread_start_us":94,"threads_started":1}
I20260925 05:16:45.044590 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.031514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33189 (local address 127.13.160.190:35237)
0925 05:16:45.034740 (+ 3226us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.034742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.034749 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:16:45.034771 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.034772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.034783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.034874 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:16:45.035611 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.036132 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.037172 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.037436 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.037518 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.040235 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:16:45.040237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:16:45.040237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:16:45.040246 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:16:45.042180 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.042712 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.042714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.042714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.042792 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.043136 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.043137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.043137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.043286 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:16:45.043349 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.044396 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.044489 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3197,"thread_start_us":99,"threads_started":1}
I20260925 05:16:45.045226 25368 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35237
I20260925 05:16:45.045367 25368 heartbeater.cc:464] Registering TS with master...
I20260925 05:16:45.045611 25368 heartbeater.cc:511] Master 127.13.160.190:35237 requested a full tablet report, sending...
I20260925 05:16:45.046221 25156 ts_manager.cc:194] Registered new tserver with Master: feab14ca93bb4d80abc68f06611a7d38 (127.13.160.129:41351)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
I20260925 05:16:45.047168 25156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:33189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:16:45.065444 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:60595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:16:45.192273 25374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:16:45.192584 25374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:16:45.192624 25374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:16:45.194448 25374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:16:45.194528 25374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:16:45.194572 25374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:16:45.196936 25374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.160.148:60595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:16:45.197244 25374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:16:45.197536 25374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 05:16:45.205199 25382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:16:45.205549 25384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:45.206184 25374 server_base.cc:1034] running on GCE node
W20260925 05:16:45.207027 25381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:45.207310 25374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:16:45.207652 25374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:16:45.208815 25374 hybrid_clock.cc:648] HybridClock initialized: now 1790313405208796 us; error 41 us; skew 500 ppm
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:16:45.212258 25374 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:16:45.212842 25374 webserver.cc:460] Webserver started at http://127.13.160.130:35487/ using document root <none> and password file <none>
I20260925 05:16:45.213272 25374 fs_manager.cc:359] Metadata directory not provided
I20260925 05:16:45.213338 25374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:16:45.213476 25374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:45.233184 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "17615202993d493bafcd5b1c736b33ff"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "fd378db848b9340dae239500aea4182c"
server_key_iv: "439cab8b23d1119e64025d6ab41e8313"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.233783 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "17615202993d493bafcd5b1c736b33ff"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "fd378db848b9340dae239500aea4182c"
server_key_iv: "439cab8b23d1119e64025d6ab41e8313"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.236464 25374 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 05:16:45.252604 25391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.254505 25374 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 05:16:45.254649 25374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "17615202993d493bafcd5b1c736b33ff"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "fd378db848b9340dae239500aea4182c"
server_key_iv: "439cab8b23d1119e64025d6ab41e8313"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.254767 25374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:16:45.276170 25374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:16:45.278772 25374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:16:45.278973 25374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:16:45.279232 25374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:16:45.279608 25374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:16:45.279657 25374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.279695 25374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:16:45.279719 25374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.286449 25374 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:36353
I20260925 05:16:45.289456 25504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:36353 every 8 connection(s)
I20260925 05:16:45.289619 25374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:16:45.293699 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 25374
I20260925 05:16:45.293848 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:16:45.294042 13954 external_mini_cluster.cc:1428] Setting key d71da79262931e278409bf2a848e3206
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:16:45.302330 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.290626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35237 (local address 127.13.160.130:43173)
0925 05:16:45.291054 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.291104 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.291755 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.292146 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.292151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.292422 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:16:45.292712 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.292719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.293753 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.293754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.295096 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.295099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.295635 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.295636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.295801 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.296450 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:16:45.296463 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:16:45.298254 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:16:45.300502 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.300511 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.300514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.300832 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.301084 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.301085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.301085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.301249 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.301572 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:16:45.301578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:16:45.301762 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.301890 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.302068 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":86,"threads_started":1}
I20260925 05:16:45.302783 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.290955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:43173 (local address 127.13.160.190:35237)
0925 05:16:45.291009 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.291012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.291189 (+ 177us) server_negotiation.cc:407] Connection header received
0925 05:16:45.291896 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.291898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.291914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.291946 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:16:45.292828 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.293621 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.295200 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.295524 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.295605 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.298389 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:16:45.298391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:16:45.298391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:16:45.298400 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:16:45.300405 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.300943 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.300945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.300945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.301010 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.301325 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.301326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.301327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.301483 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:16:45.301544 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.302592 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.302690 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:16:45.303185 25505 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35237
I20260925 05:16:45.303283 25505 heartbeater.cc:464] Registering TS with master...
I20260925 05:16:45.303486 25505 heartbeater.cc:511] Master 127.13.160.190:35237 requested a full tablet report, sending...
I20260925 05:16:45.303838 25156 ts_manager.cc:194] Registered new tserver with Master: 17615202993d493bafcd5b1c736b33ff (127.13.160.130:36353)
I20260925 05:16:45.304332 25156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:43173
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:16:45.334658 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:60595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:16:45.448364 25511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:16:45.448628 25511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:16:45.448657 25511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:16:45.450098 25511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:16:45.450152 25511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:16:45.450182 25511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:16:45.452036 25511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.160.148:60595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:35237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:16:45.452258 25511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:16:45.452466 25511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:16:45.457571 25516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:45.458236 25511 server_base.cc:1034] running on GCE node
W20260925 05:16:45.463873 25517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:16:45.463896 25519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:16:45.464232 25511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:16:45.464551 25511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:16:45.465723 25511 hybrid_clock.cc:648] HybridClock initialized: now 1790313405465701 us; error 39 us; skew 500 ppm
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:16:45.468888 25511 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:16:45.469442 25511 webserver.cc:460] Webserver started at http://127.13.160.131:32895/ using document root <none> and password file <none>
I20260925 05:16:45.469702 25511 fs_manager.cc:359] Metadata directory not provided
I20260925 05:16:45.469770 25511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:16:45.469934 25511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:45.484859 25511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "922c3eb5053148b38e3d87e3c33146a2"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "087dbcff8e90e4995c0c213a324eb5a3"
server_key_iv: "4f9590a7c5ddf29e9dd170a3def1b5b3"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.485841 25511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "922c3eb5053148b38e3d87e3c33146a2"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "087dbcff8e90e4995c0c213a324eb5a3"
server_key_iv: "4f9590a7c5ddf29e9dd170a3def1b5b3"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.490455 25511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260925 05:16:45.508283 25526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.508586 25511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 05:16:45.508765 25511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "922c3eb5053148b38e3d87e3c33146a2"
format_stamp: "Formatted at 2026-09-25 05:16:45 on dist-test-slave-9tgz"
server_key: "087dbcff8e90e4995c0c213a324eb5a3"
server_key_iv: "4f9590a7c5ddf29e9dd170a3def1b5b3"
server_key_version: "kuduclusterkey@0"
I20260925 05:16:45.508941 25511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:16:45.533574 25511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:16:45.536340 25511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:16:45.536590 25511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:16:45.536922 25511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:16:45.537395 25511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:16:45.537493 25511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.537572 25511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:16:45.537640 25511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:16:45.544648 25511 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:44897
I20260925 05:16:45.545187 25511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:16:45.550500 25639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:44897 every 8 connection(s)
I20260925 05:16:45.552726 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 25511
I20260925 05:16:45.552835 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:16:45.553016 13954 external_mini_cluster.cc:1428] Setting key 225796d5a4baceb376260b1018649f89
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:16:45.561959 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.548887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:58561 (local address 127.13.160.190:35237)
0925 05:16:45.548938 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.548941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.549407 (+ 466us) server_negotiation.cc:407] Connection header received
0925 05:16:45.550129 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.550131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.550145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.550245 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:16:45.551251 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.552065 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.555107 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.555432 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.555481 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.558403 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:16:45.558405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:16:45.558406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:16:45.558415 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:16:45.560262 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.560809 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.560810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.560811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.560875 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.561166 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.561166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.561167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.561313 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:16:45.561370 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.561764 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.561852 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:16:45.562793 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.548564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:35237 (local address 127.13.160.131:58561)
0925 05:16:45.549225 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.549280 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.550001 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.550410 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.550416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.550698 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:16:45.551124 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.551132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.552195 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.552196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.555006 (+ 2810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.555009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.555624 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.555625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.555777 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.556454 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:16:45.556465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:16:45.558259 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:16:45.560377 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.560386 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.560390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.560716 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.560947 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:16:45.560948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:16:45.560948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:16:45.561095 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:16:45.561417 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:16:45.561424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:16:45.561608 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.562356 (+ 748us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.562540 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":87,"threads_started":1}
I20260925 05:16:45.563444 25640 heartbeater.cc:347] Connected to a master server at 127.13.160.190:35237
I20260925 05:16:45.563553 25640 heartbeater.cc:464] Registering TS with master...
I20260925 05:16:45.563771 25640 heartbeater.cc:511] Master 127.13.160.190:35237 requested a full tablet report, sending...
I20260925 05:16:45.564168 25156 ts_manager.cc:194] Registered new tserver with Master: 922c3eb5053148b38e3d87e3c33146a2 (127.13.160.131:44897)
I20260925 05:16:45.564695 25156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:58561
I20260925 05:16:45.565654 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:16:45.575910 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.567450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.13.160.190:35237)
0925 05:16:45.567494 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.567496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.567674 (+ 178us) server_negotiation.cc:407] Connection header received
0925 05:16:45.567745 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.567746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.567758 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.567817 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:16:45.568615 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.569380 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.570481 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.570768 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.570816 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.572608 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:16:45.572610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:16:45.572611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:16:45.572618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:16:45.574558 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.575014 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.575016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.575016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.575077 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:16:45.575283 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:16:45.575284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:16:45.575285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:16:45.575432 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:16:45.575508 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.575727 (+ 219us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.575809 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:16:45.578199 25156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:16:45.580689 25156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:16:45.591936 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.585627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.13.160.131:44897)
0925 05:16:45.586228 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.586234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.586244 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:16:45.586282 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.586284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.586301 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.586391 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:16:45.586965 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.588076 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.590096 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.591343 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.591425 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.591493 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.591794 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.591839 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":88,"threads_started":1}
I20260925 05:16:45.592511 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.585313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:44897 (local address 127.0.0.1:55720)
0925 05:16:45.585828 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.585875 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.585945 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.586430 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.586434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.586437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:16:45.586521 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.586523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.588193 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.588194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.589964 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.589968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.591562 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.591562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.591659 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.591676 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.592346 (+ 670us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.592431 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":83,"threads_started":1}
I20260925 05:16:45.593516 25574 tablet_service.cc:1467] Processing CreateTablet for tablet bd744cd822a547a79676597e8ef0551d (DEFAULT_TABLE table=test-table [id=a36e15974e1f42b0ac7dc58ca021b104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:16:45.593945 25574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd744cd822a547a79676597e8ef0551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:16:45.597913 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.585367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:41351 (local address 127.0.0.1:59568)
0925 05:16:45.586064 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.586072 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.586106 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.586807 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.586808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.586810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:16:45.586856 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.586858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.592492 (+ 5634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.592493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.593639 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.593642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.597632 (+ 3990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.597633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.597722 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.597728 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.597764 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.597816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":95,"threads_started":1}
I20260925 05:16:45.599231 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.585880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.13.160.129:41351)
0925 05:16:45.586646 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.586651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.586660 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:16:45.586693 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.586695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.586711 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.591077 (+ 4366us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:16:45.591162 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.592234 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.593886 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.594690 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.598240 (+ 3550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.599056 (+ 816us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.599090 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.599148 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":59,"threads_started":1}
I20260925 05:16:45.599800 25658 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Bootstrap starting.
I20260925 05:16:45.600116 25289 tablet_service.cc:1467] Processing CreateTablet for tablet bd744cd822a547a79676597e8ef0551d (DEFAULT_TABLE table=test-table [id=a36e15974e1f42b0ac7dc58ca021b104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:16:45.600503 25289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd744cd822a547a79676597e8ef0551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:16:45.601635 25658 tablet_bootstrap.cc:654] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Neither blocks nor log segments found. Creating new log.
I20260925 05:16:45.602480 25658 log.cc:824] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Log is configured to *not* fsync() on all Append() calls
I20260925 05:16:45.603948 25660 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: Bootstrap starting.
I20260925 05:16:45.604736 25660 tablet_bootstrap.cc:654] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: Neither blocks nor log segments found. Creating new log.
I20260925 05:16:45.605121 25660 log.cc:824] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: Log is configured to *not* fsync() on all Append() calls
I20260925 05:16:45.606374 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.585599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:36353 (local address 127.0.0.1:58828)
0925 05:16:45.586499 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.586508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.586547 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.595077 (+ 8530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.595078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.595081 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:16:45.595127 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.595128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.599162 (+ 4034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.599163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.604905 (+ 5742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.604911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.606103 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.606104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.606179 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.606186 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.606219 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.606275 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":72,"threads_started":1}
I20260925 05:16:45.606825 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.586019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.13.160.130:36353)
0925 05:16:45.594859 (+ 8840us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.594864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.594874 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:16:45.594943 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.594946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.594964 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.595202 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:16:45.595701 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.599052 (+ 3351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.605071 (+ 6019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.605984 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.606590 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.606672 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.606702 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.606745 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8790,"thread_start_us":71,"threads_started":1}
I20260925 05:16:45.607579 25439 tablet_service.cc:1467] Processing CreateTablet for tablet bd744cd822a547a79676597e8ef0551d (DEFAULT_TABLE table=test-table [id=a36e15974e1f42b0ac7dc58ca021b104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:16:45.607963 25439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd744cd822a547a79676597e8ef0551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:16:45.612097 25662 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Bootstrap starting.
I20260925 05:16:45.613140 25662 tablet_bootstrap.cc:654] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Neither blocks nor log segments found. Creating new log.
I20260925 05:16:45.613497 25662 log.cc:824] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Log is configured to *not* fsync() on all Append() calls
I20260925 05:16:45.614449 25662 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: No bootstrap required, opened a new log
I20260925 05:16:45.614529 25662 ts_tablet_manager.cc:1397] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:16:45.615526 25658 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: No bootstrap required, opened a new log
I20260925 05:16:45.615605 25658 ts_tablet_manager.cc:1397] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260925 05:16:45.615727 25660 tablet_bootstrap.cc:492] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: No bootstrap required, opened a new log
I20260925 05:16:45.615835 25660 ts_tablet_manager.cc:1397] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260925 05:16:45.617751 25658 raft_consensus.cc:348] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.617955 25658 raft_consensus.cc:374] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:16:45.617993 25658 raft_consensus.cc:729] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c3eb5053148b38e3d87e3c33146a2, State: Initialized, Role: FOLLOWER
I20260925 05:16:45.618134 25658 consensus_queue.cc:260] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.618448 25658 ts_tablet_manager.cc:1428] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:16:45.618594 25640 heartbeater.cc:503] Master 127.13.160.190:35237 was elected leader, sending a full tablet report...
I20260925 05:16:45.619315 25660 raft_consensus.cc:348] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.619576 25660 raft_consensus.cc:374] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:16:45.619650 25660 raft_consensus.cc:729] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feab14ca93bb4d80abc68f06611a7d38, State: Initialized, Role: FOLLOWER
I20260925 05:16:45.619828 25660 consensus_queue.cc:260] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.620141 25660 ts_tablet_manager.cc:1428] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 05:16:45.620285 25368 heartbeater.cc:503] Master 127.13.160.190:35237 was elected leader, sending a full tablet report...
I20260925 05:16:45.621562 25662 raft_consensus.cc:348] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.621759 25662 raft_consensus.cc:374] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:16:45.621797 25662 raft_consensus.cc:729] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17615202993d493bafcd5b1c736b33ff, State: Initialized, Role: FOLLOWER
I20260925 05:16:45.621924 25662 consensus_queue.cc:260] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.622215 25662 ts_tablet_manager.cc:1428] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20260925 05:16:45.622257 25505 heartbeater.cc:503] Master 127.13.160.190:35237 was elected leader, sending a full tablet report...
I20260925 05:16:45.626957 25664 raft_consensus.cc:482] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:16:45.627092 25664 raft_consensus.cc:504] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.631385 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.627605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:41351 (local address 127.13.160.131:54079)
0925 05:16:45.627703 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.627716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.627754 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.628035 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.628036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.628038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:16:45.628111 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.628113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.629009 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.629010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.630237 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.630240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.631131 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.631133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.631214 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.631218 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.631262 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.631301 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 05:16:45.631688 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.627844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:54079 (local address 127.13.160.129:41351)
0925 05:16:45.627897 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.627899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.627906 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:16:45.627930 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.627931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.627942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.627988 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:16:45.628177 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.628914 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.630390 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.631019 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.631087 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.631097 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.631581 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.631626 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:16:45.631964 25312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd744cd822a547a79676597e8ef0551d" candidate_uuid: "922c3eb5053148b38e3d87e3c33146a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feab14ca93bb4d80abc68f06611a7d38" is_pre_election: true
I20260925 05:16:45.632153 25312 raft_consensus.cc:2454] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c3eb5053148b38e3d87e3c33146a2 in term 0.
I20260925 05:16:45.632504 25529 leader_election.cc:302] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 922c3eb5053148b38e3d87e3c33146a2, feab14ca93bb4d80abc68f06611a7d38; no voters:
I20260925 05:16:45.632778 25667 raft_consensus.cc:2781] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:16:45.632876 25667 raft_consensus.cc:482] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:16:45.632908 25667 raft_consensus.cc:3037] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:16:45.633848 25667 raft_consensus.cc:504] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.634152 25312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd744cd822a547a79676597e8ef0551d" candidate_uuid: "922c3eb5053148b38e3d87e3c33146a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feab14ca93bb4d80abc68f06611a7d38"
I20260925 05:16:45.634234 25312 raft_consensus.cc:3037] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:16:45.635223 25312 raft_consensus.cc:2454] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c3eb5053148b38e3d87e3c33146a2 in term 1.
I20260925 05:16:45.635406 25529 leader_election.cc:302] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 922c3eb5053148b38e3d87e3c33146a2, feab14ca93bb4d80abc68f06611a7d38; no voters:
I20260925 05:16:45.635618 25667 leader_election.cc:288] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [CANDIDATE]: Term 1 election: Requested vote from peers feab14ca93bb4d80abc68f06611a7d38 (127.13.160.129:41351), 17615202993d493bafcd5b1c736b33ff (127.13.160.130:36353)
I20260925 05:16:45.635658 25667 raft_consensus.cc:2781] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:16:45.635692 25667 raft_consensus.cc:686] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 1 LEADER]: Becoming Leader. State: Replica: 922c3eb5053148b38e3d87e3c33146a2, State: Running, Role: LEADER
I20260925 05:16:45.635779 25667 consensus_queue.cc:237] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } }
I20260925 05:16:45.636631 25155 catalog_manager.cc:5495] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 922c3eb5053148b38e3d87e3c33146a2 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "922c3eb5053148b38e3d87e3c33146a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922c3eb5053148b38e3d87e3c33146a2" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 } health_report { overall_health: UNKNOWN } } }
I20260925 05:16:45.636833 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.632202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:36353 (local address 127.13.160.131:55509)
0925 05:16:45.632866 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.632878 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:16:45.632938 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:16:45.633177 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:16:45.633178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:16:45.633179 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:16:45.633214 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.633215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.634197 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.634198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.635290 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:16:45.635292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.636610 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.636611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:16:45.636678 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.636682 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:16:45.636729 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 05:16:45.636765 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645}
I20260925 05:16:45.637100 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.632320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:55509 (local address 127.13.160.130:36353)
0925 05:16:45.632401 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.632404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.632989 (+ 585us) server_negotiation.cc:407] Connection header received
0925 05:16:45.633016 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.633017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.633030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.633071 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:16:45.633282 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.634020 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.635447 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.636036 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.636103 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.636112 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.636992 (+ 880us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.637038 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 05:16:45.637295 25664 leader_election.cc:288] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers feab14ca93bb4d80abc68f06611a7d38 (127.13.160.129:41351), 17615202993d493bafcd5b1c736b33ff (127.13.160.130:36353)
I20260925 05:16:45.637344 25456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd744cd822a547a79676597e8ef0551d" candidate_uuid: "922c3eb5053148b38e3d87e3c33146a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17615202993d493bafcd5b1c736b33ff" is_pre_election: true
I20260925 05:16:45.637524 25456 raft_consensus.cc:2454] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922c3eb5053148b38e3d87e3c33146a2 in term 0.
I20260925 05:16:45.637650 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd744cd822a547a79676597e8ef0551d" candidate_uuid: "922c3eb5053148b38e3d87e3c33146a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17615202993d493bafcd5b1c736b33ff"
I20260925 05:16:45.637718 25455 raft_consensus.cc:3037] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:16:45.638368 25455 raft_consensus.cc:2454] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922c3eb5053148b38e3d87e3c33146a2 in term 1.
I20260925 05:16:45.654325 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.650930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.13.160.131:44897)
0925 05:16:45.650973 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:16:45.650975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:16:45.650985 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:16:45.651075 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:16:45.651076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:16:45.651097 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:16:45.651178 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:16:45.651347 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.652102 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.653249 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:16:45.653572 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:16:45.653643 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:16:45.653848 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:16:45.654002 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:16:45.654084 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:16:45.654193 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 05:16:45.654220 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:16:45.656502 25455 raft_consensus.cc:1261] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 1 FOLLOWER]: Refusing update from remote peer 922c3eb5053148b38e3d87e3c33146a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:16:45.656934 25312 raft_consensus.cc:1261] T bd744cd822a547a79676597e8ef0551d P feab14ca93bb4d80abc68f06611a7d38 [term 1 FOLLOWER]: Refusing update from remote peer 922c3eb5053148b38e3d87e3c33146a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:16:45.656913 25668 consensus_queue.cc:1035] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17615202993d493bafcd5b1c736b33ff" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:16:45.657334 25667 consensus_queue.cc:1035] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feab14ca93bb4d80abc68f06611a7d38" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 41351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:16:45.659921 25671 mvcc.cc:205] Tried to move back new op lower bound from 7333123709565689856 to 7333123709484896256. Current Snapshot: MvccSnapshot[applied={T|T < 7333123709565689856}]
I20260925 05:16:45.661417 25673 mvcc.cc:205] Tried to move back new op lower bound from 7333123709565689856 to 7333123709484896256. Current Snapshot: MvccSnapshot[applied={T|T < 7333123709565689856}]
I20260925 05:16:45.662725 25672 mvcc.cc:205] Tried to move back new op lower bound from 7333123709565689856 to 7333123709484896256. Current Snapshot: MvccSnapshot[applied={T|T < 7333123709565689856 or (T in {7333123709565689856})}]
I20260925 05:16:45.663923 25155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:16:45.664022 25155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:16:45.664832 25155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b351e8bb1e64ce5b1053f9285f1afde: Sending DeleteTablet for 3 replicas of tablet bd744cd822a547a79676597e8ef0551d
I20260925 05:16:45.665275 25289 tablet_service.cc:1514] Processing DeleteTablet for tablet bd744cd822a547a79676597e8ef0551d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:45 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:59568
I20260925 05:16:45.665515 25439 tablet_service.cc:1514] Processing DeleteTablet for tablet bd744cd822a547a79676597e8ef0551d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:45 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:58828
I20260925 05:16:45.665911 25574 tablet_service.cc:1514] Processing DeleteTablet for tablet bd744cd822a547a79676597e8ef0551d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:16:45 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:55720
I20260925 05:16:45.666575 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 25237
I20260925 05:16:45.669116 25681 tablet_replica.cc:330] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: stopping tablet replica
I20260925 05:16:45.669284 25681 raft_consensus.cc:2229] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:16:45.669484 25681 raft_consensus.cc:2258] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:16:45.669693 25682 tablet_replica.cc:330] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: stopping tablet replica
I20260925 05:16:45.670207 25681 ts_tablet_manager.cc:1905] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:16:45.670357 25682 raft_consensus.cc:2229] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:16:45.670512 25682 raft_consensus.cc:2258] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:16:45.671304 25682 ts_tablet_manager.cc:1905] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:16:45.674111 25681 ts_tablet_manager.cc:1918] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:16:45.674785 25681 log.cc:1196] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/bd744cd822a547a79676597e8ef0551d
I20260925 05:16:45.675513 25681 ts_tablet_manager.cc:1939] T bd744cd822a547a79676597e8ef0551d P 17615202993d493bafcd5b1c736b33ff: Deleting consensus metadata
I20260925 05:16:45.676100 25140 catalog_manager.cc:4841] TS 17615202993d493bafcd5b1c736b33ff (127.13.160.130:36353): tablet bd744cd822a547a79676597e8ef0551d (table test-table [id=a36e15974e1f42b0ac7dc58ca021b104]) successfully deleted
I20260925 05:16:45.676479 25682 ts_tablet_manager.cc:1918] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:16:45.676612 25682 log.cc:1196] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/bd744cd822a547a79676597e8ef0551d
I20260925 05:16:45.676959 25682 ts_tablet_manager.cc:1939] T bd744cd822a547a79676597e8ef0551d P 922c3eb5053148b38e3d87e3c33146a2: Deleting consensus metadata
I20260925 05:16:45.677426 25141 catalog_manager.cc:4841] TS 922c3eb5053148b38e3d87e3c33146a2 (127.13.160.131:44897): tablet bd744cd822a547a79676597e8ef0551d (table test-table [id=a36e15974e1f42b0ac7dc58ca021b104]) successfully deleted
W20260925 05:16:45.681630 25142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:41351 (error 108)
I20260925 05:16:45.682641 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:16:45.682522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:41351 (local address 127.0.0.1:59576)
0925 05:16:45.682570 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 05:16:45.682600 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 05:16:45.682869 25142 catalog_manager.cc:4584] TS feab14ca93bb4d80abc68f06611a7d38 (127.13.160.129:41351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd744cd822a547a79676597e8ef0551d: Network error: Client connection negotiation failed: client connection to 127.13.160.129:41351: connect: Connection refused (error 111)
I20260925 05:16:45.683195 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 25374
I20260925 05:16:45.712376 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 25511
Sep 25 05:16:45 dist-test-slave-9tgz krb5kdc[24680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313405, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:16:45.720620 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 25124
I20260925 05:16:45.726687 13954 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 05:16:46.061648 13954 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 05:16:46.061755 13954 mini_ranger.cc:66] Stopping Ranger...
I20260925 05:16:46.406721 13954 mini_ranger.cc:68] Stopped Ranger
2026-09-25 05:16:46.407 UTC [24710] LOG: received smart shutdown request
2026-09-25 05:16:46.410 UTC [24710] LOG: background worker "logical replication launcher" (PID 24719) exited with exit code 1
2026-09-25 05:16:46.410 UTC [24714] LOG: shutting down
2026-09-25 05:16:46.573 UTC [24710] LOG: database system is shut down
2026-09-25T05:16:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46653 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:16:46 dist-test-slave-9tgz krb5kdc[25692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:16:46 dist-test-slave-9tgz krb5kdc[25692](info): set up 2 sockets
Sep 25 05:16:46 dist-test-slave-9tgz krb5kdc[25692](info): commencing operation
krb5kdc: starting...
W20260925 05:16:47.719549 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:16:47 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:16:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangeradmin_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangeradmin_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangerlookup_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangerlookup_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
I20260925 05:16:47.879351 13954 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskq3iYZX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/postgres -l logfile start
2026-09-25 05:16:50.319 UTC [25722] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 05:16:50.319 UTC [25722] LOG: listening on IPv4 address "127.13.160.148", port 57421
2026-09-25 05:16:50.323 UTC [25722] LOG: listening on Unix socket "/tmp/.s.PGSQL.57421"
2026-09-25 05:16:50.350 UTC [25725] LOG: database system was shut down at 2026-09-25 05:16:48 UTC
2026-09-25 05:16:50.354 UTC [25722] LOG: database system is ready to accept connections
I20260925 05:16:51.307693 13954 mini_postgres.cc:96] Postgres bound to 57421
2026-09-25 05:16:51.313 UTC [25733] FATAL: database "slave" does not exist
127.13.160.148:57421 - accepting connections
I20260925 05:16:51.314407 13954 mini_ranger.cc:161] Starting Ranger...
I20260925 05:16:51.329646 13954 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 05:16:51.785761 13954 mini_ranger.cc:87] Created ranger Postgres database
I20260925 05:16:51.785856 13954 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin
2026-09-25 05:16:52,061 [I] DB FLAVOR :POSTGRES
2026-09-25 05:16:52,062 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 05:16:52,062 [I] Checking connection..
2026-09-25 05:16:52,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 05:16:52,311 [I] Checking connection passed.
2026-09-25 05:16:52,311 [I] --------- Verifying version history table ---------
2026-09-25 05:16:52,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:52,560 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 05:16:52,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:52,824 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 05:16:52,824 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 05:16:52,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 05:16:53.088 UTC [25797] WARNING: there is no transaction in progress
2026-09-25 05:16:53,095 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 05:16:53,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 05:16:53,348 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 05:16:53,349 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 05:16:53,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 05:16:53,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 05:16:53,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9tgz.c.gcp-upstream.internal','N') ;"
2026-09-25 05:16:54,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 05:16:54,302 [I] Table x_portal_user does not exist in database ranger
2026-09-25 05:16:54,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 05:16:54,527 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 05:16:54,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 05:16:54,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 05:16:55,044 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 05:16:55,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 05:16:56.051 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.068 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.085 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.100 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.119 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.141 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.203 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.210 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.224 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.235 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.246 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.259 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.269 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.275 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.282 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56.291 UTC [25905] WARNING: there is no transaction in progress
2026-09-25 05:16:56,963 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 05:16:56,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 05:16:57,217 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 05:16:57,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 05:16:57,464 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 05:16:57,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 05:16:57,723 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 05:16:57,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 05:16:57,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 05:16:58,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9tgz.c.gcp-upstream.internal';"
2026-09-25 05:16:58,452 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 05:16:58,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 05:16:58,682 [I] DB_PATCHES have already been applied
I20260925 05:16:58.688271 13954 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskq3iYZX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 05:16:58.688357 13954 mini_ranger.cc:193] Using host: 127.13.160.148
I20260925 05:16:58.691465 13954 mini_ranger.cc:227] Ranger admin URL: http://127.13.160.148:35883
Sep 25, 2026 5:16:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 5:16:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 5:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 5:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 5:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 5:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 5:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 05:16:59.822110 13954 mini_ranger.cc:160] Time spent starting Ranger: real 8.508s user 0.000s sys 0.009s
Sep 25, 2026 5:17:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 5:17:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 5:17:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 5:17:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 5:17:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 05:17:21.143769 13954 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.160.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.160.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangerkms_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.160.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/rangerkms_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.160.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.160.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.160.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/HTTP_127.13.160.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 05:17:21 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 05:17:21.226843 13954 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 05:17:21.236719 13954 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 05:17:22.081390 13954 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 05:17:22.081519 13954 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms
I20260925 05:17:22.081553 13954 mini_ranger_kms.cc:227] Using postgres at 127.13.160.148:57421
2026-09-25 05:17:22,276 [I] DB FLAVOR :POSTGRES
2026-09-25 05:17:22,276 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 05:17:22,276 [I] Checking connection
2026-09-25 05:17:22,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 05:17:22,577 [I] connection success
2026-09-25 05:17:22,577 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 05:17:22,577 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 05:17:22,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 05:17:22,856 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 05:17:22,857 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 05:17:22,857 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 05:17:22,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.160.148:57421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 05:17:23,203 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 05:17:23.702760 13954 mini_ranger_kms.cc:310] Created kms service
I20260925 05:17:23.911116 13954 mini_ranger_kms.cc:326] Created kudu user
I20260925 05:17:23.954577 13954 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 05:17:23 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:24.333360 13954 mini_ranger_kms.cc:384] Added ranger policy
I20260925 05:17:24.333473 13954 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms:/tmp/dist-test-taskq3iYZX/build/release/bin/postgresql.jar:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskq3iYZX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 05:17:24.333492 13954 mini_ranger_kms.cc:240] Using host: 127.13.160.148
I20260925 05:17:24.336673 13954 mini_ranger_kms.cc:276] Ranger KMS PID: 26085
I20260925 05:17:24.336762 13954 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.160.148:38539
Sep 25, 2026 5:17:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 5:17:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 5:17:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 5:17:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 5:17:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 5:17:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 05:17:26.643679 13954 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.417s user 0.006s sys 0.170s
I20260925 05:17:26.643780 13954 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 05:17:26.643823 13954 mini_ranger_kms.cc:401] 127.13.160.148:38539/kms/v1/keys
Sep 25 05:17:26 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790313446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 05:17:27 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
Sep 25 05:17:27 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313446, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.160.148@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
Sep 25 05:17:30 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:17:30.093649 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40291
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40291
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:30.211328 26136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:30.211565 26136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:30.211588 26136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:30.212792 26136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:30.212837 26136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:30.212852 26136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:30.212865 26136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:30.212877 26136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:30.214502 26136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.160.148:38539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40291
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:30.214717 26136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:30.214994 26136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:30.220070 26144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:30.220070 26142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:30.220090 26141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:30.220126 26136 server_base.cc:1034] running on GCE node
I20260925 05:17:30.220567 26136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:30.220849 26136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:30.222029 26136 hybrid_clock.cc:648] HybridClock initialized: now 1790313450222024 us; error 43 us; skew 500 ppm
Sep 25 05:17:30 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:30.230262 26136 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:30.230792 26136 webserver.cc:460] Webserver started at http://127.13.160.190:40863/ using document root <none> and password file <none>
I20260925 05:17:30.231094 26136 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:30.231163 26136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:30.231328 26136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:17:30 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313450, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:30.280172 26136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "7d5ad27cfc0047f082d81e21956afe7f"
format_stamp: "Formatted at 2026-09-25 05:17:30 on dist-test-slave-9tgz"
server_key: "18d340d6f024ad8eb654f4041a2f8a11"
server_key_iv: "c69469d67c1bee35840a1f097543a164"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:30.280637 26136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "7d5ad27cfc0047f082d81e21956afe7f"
format_stamp: "Formatted at 2026-09-25 05:17:30 on dist-test-slave-9tgz"
server_key: "18d340d6f024ad8eb654f4041a2f8a11"
server_key_iv: "c69469d67c1bee35840a1f097543a164"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:30.282215 26136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:17:30.297145 26151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:30.297334 26136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:30.297425 26136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "7d5ad27cfc0047f082d81e21956afe7f"
format_stamp: "Formatted at 2026-09-25 05:17:30 on dist-test-slave-9tgz"
server_key: "18d340d6f024ad8eb654f4041a2f8a11"
server_key_iv: "c69469d67c1bee35840a1f097543a164"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:30.297526 26136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:30.318861 26136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:30.321188 26136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:30.321352 26136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:30.326448 26136 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40291
I20260925 05:17:30.326479 26203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40291 every 8 connection(s)
I20260925 05:17:30.326964 26136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:30.327746 26204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:30.330191 26204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Bootstrap starting.
I20260925 05:17:30.330796 26204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:30.331152 26204 log.cc:824] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:30.331370 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26136
I20260925 05:17:30.331490 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:17:30.331676 13954 external_mini_cluster.cc:1428] Setting key 32f96afcda0e87a49c7ede2e3005a03b
I20260925 05:17:30.331995 26204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: No bootstrap required, opened a new log
I20260925 05:17:30.334815 26204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:30.335047 26204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:30.335090 26204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5ad27cfc0047f082d81e21956afe7f, State: Initialized, Role: FOLLOWER
I20260925 05:17:30.335271 26204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:30.335357 26204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:30.335387 26204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:30.335466 26204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:30.336207 26204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:30.336335 26204 leader_election.cc:302] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d5ad27cfc0047f082d81e21956afe7f; no voters:
I20260925 05:17:30.336544 26204 leader_election.cc:288] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:17:30.336570 26209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:30.336790 26209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 LEADER]: Becoming Leader. State: Replica: 7d5ad27cfc0047f082d81e21956afe7f, State: Running, Role: LEADER
I20260925 05:17:30.336900 26204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:30.336928 26209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:30.338340 26209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d5ad27cfc0047f082d81e21956afe7f. Latest consensus state: current_term: 1 leader_uuid: "7d5ad27cfc0047f082d81e21956afe7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } } }
I20260925 05:17:30.338491 26209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:30.338646 26210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d5ad27cfc0047f082d81e21956afe7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } } }
I20260925 05:17:30.338737 26210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:30.339098 26204 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 05:17:30.339134 26223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 05:17:30 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:30.341619 26223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:30.346128 26223 catalog_manager.cc:1347] Generated new cluster ID: 3e096dadfb95431996af3eab2099e2a6
I20260925 05:17:30.346186 26223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:17:30.348318 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:30.332438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.13.160.190:40291)
0925 05:17:30.332627 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:30.332637 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:30.332676 (+ 39us) server_negotiation.cc:407] Connection header received
0925 05:17:30.333443 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:30.333453 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:30.333521 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:30.333677 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:30.334491 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:30.335950 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:30.337092 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:30.337802 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:30.337948 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:30.342149 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:30.342193 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:30.342194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:30.342282 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:30.345747 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:30.346264 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:30.346268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:30.346268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:30.346405 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:30.346637 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:30.346638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:30.346638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:30.347012 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:30.347148 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:30.347661 (+ 513us) server_negotiation.cc:299] Negotiation successful
0925 05:17:30.348012 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260925 05:17:30.374022 26223 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:17:30.374926 26223 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:30.388341 26223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Generated new TSK 0
I20260925 05:17:30.388717 26223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:17:31.074927 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:31.188738 26249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:31.188998 26249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:31.189042 26249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:31.190297 26249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:31.190387 26249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:31.190451 26249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:31.192293 26249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:31.192587 26249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:31.192864 26249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:31.197932 26257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:31.198037 26255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:31.197948 26254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:31.198868 26249 server_base.cc:1034] running on GCE node
I20260925 05:17:31.199131 26249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:31.199397 26249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:31.202653 26249 hybrid_clock.cc:648] HybridClock initialized: now 1790313451202633 us; error 27 us; skew 500 ppm
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:31.205554 26249 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:31.206091 26249 webserver.cc:460] Webserver started at http://127.13.160.129:32823/ using document root <none> and password file <none>
I20260925 05:17:31.206393 26249 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:31.206444 26249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:31.206586 26249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:31.221657 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "479ef5b41bf44cb8a19afdbb559ee7e3"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "3be0f6f24fb0175dbeec6eb1108545ba"
server_key_iv: "20cd88f6b1a6d774c15c8092d5b2c050"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.222188 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "479ef5b41bf44cb8a19afdbb559ee7e3"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "3be0f6f24fb0175dbeec6eb1108545ba"
server_key_iv: "20cd88f6b1a6d774c15c8092d5b2c050"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.224126 26249 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260925 05:17:31.242372 26265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.242782 26249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:31.242944 26249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "479ef5b41bf44cb8a19afdbb559ee7e3"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "3be0f6f24fb0175dbeec6eb1108545ba"
server_key_iv: "20cd88f6b1a6d774c15c8092d5b2c050"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.243057 26249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:31.262682 26249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:31.265463 26249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:31.265729 26249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:31.266064 26249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:31.266544 26249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:31.266640 26249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.266739 26249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:31.266813 26249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.273157 26249 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40547
I20260925 05:17:31.273713 26249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:17:31.275305 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26249
I20260925 05:17:31.275410 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:17:31.275605 13954 external_mini_cluster.cc:1428] Setting key 11cadcd8659a3d7794c6449b3aaf6f90
I20260925 05:17:31.279047 26379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40547 every 8 connection(s)
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
I20260925 05:17:31.304280 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.279815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.129:37413)
0925 05:17:31.287063 (+ 7248us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:31.287171 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:31.287910 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:31.288241 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:31.288246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:31.288499 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:31.288801 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.288808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.289886 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.289887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.291259 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.291262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.291845 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.291846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.292016 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.292690 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:31.292701 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:31.299998 (+ 7297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:31.302289 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.302303 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.302307 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.302674 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.302942 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.302943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.302943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.303113 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.303449 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:31.303457 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:31.303655 (+ 198us) client_negotiation.cc:769] Sending connection context
0925 05:17:31.303777 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 05:17:31.303987 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7172,"thread_start_us":108,"threads_started":1}
I20260925 05:17:31.304931 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.280152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:37413 (local address 127.13.160.190:40291)
0925 05:17:31.287046 (+ 6894us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:31.287049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:31.287989 (+ 940us) server_negotiation.cc:407] Connection header received
0925 05:17:31.288031 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:31.288033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:31.288050 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:31.288158 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:31.288946 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.289765 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.291365 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.291694 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.291759 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.300173 (+ 8414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:31.300176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:31.300176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:31.300187 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:31.302162 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.302770 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.302771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.302772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.302841 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.303190 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.303191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.303191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.303345 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:31.303417 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:31.304745 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0925 05:17:31.304843 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6853,"thread_start_us":93,"threads_started":1}
I20260925 05:17:31.305694 26380 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:31.305827 26380 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:31.306062 26380 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:31.306804 26168 ts_manager.cc:194] Registered new tserver with Master: 479ef5b41bf44cb8a19afdbb559ee7e3 (127.13.160.129:40547)
I20260925 05:17:31.307588 26168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:37413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:17:31.319923 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:31.446553 26386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:31.446947 26386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:31.447057 26386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:31.449257 26386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:31.449522 26386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:31.449630 26386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:31.452133 26386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:31.452685 26386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:31.453069 26386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:31.459939 26394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:31.460402 26396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:31.461736 26386 server_base.cc:1034] running on GCE node
W20260925 05:17:31.463362 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:31.463709 26386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:31.464027 26386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:31.467870 26386 hybrid_clock.cc:648] HybridClock initialized: now 1790313451467841 us; error 57 us; skew 500 ppm
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:31.471925 26386 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:31.472563 26386 webserver.cc:460] Webserver started at http://127.13.160.130:45843/ using document root <none> and password file <none>
I20260925 05:17:31.472918 26386 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:31.473026 26386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:31.473204 26386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:31.492528 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "75b8cdf1578142bf927e8d88c6dd59d0"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "f37f717c4c55690c0b8bc0a58dd3d819"
server_key_iv: "245eb8820f9eafb4bd7b5feabaa0ad34"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.494855 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "75b8cdf1578142bf927e8d88c6dd59d0"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "f37f717c4c55690c0b8bc0a58dd3d819"
server_key_iv: "245eb8820f9eafb4bd7b5feabaa0ad34"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.497009 26386 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 05:17:31.520743 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.521039 26386 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260925 05:17:31.521132 26386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "75b8cdf1578142bf927e8d88c6dd59d0"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "f37f717c4c55690c0b8bc0a58dd3d819"
server_key_iv: "245eb8820f9eafb4bd7b5feabaa0ad34"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.521232 26386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:31.572245 26386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:31.574857 26386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:31.575104 26386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:31.575369 26386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:31.575783 26386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:31.575831 26386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.575861 26386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:31.575878 26386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.582316 26386 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:43333
I20260925 05:17:31.582844 26386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:17:31.585211 26517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:43333 every 8 connection(s)
I20260925 05:17:31.587896 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26386
I20260925 05:17:31.588011 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:17:31.588184 13954 external_mini_cluster.cc:1428] Setting key d9555b56667f432621a1ea8fa7f9f233
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:31.597669 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.586551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:34035 (local address 127.13.160.190:40291)
0925 05:17:31.586619 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:31.586621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:31.586942 (+ 321us) server_negotiation.cc:407] Connection header received
0925 05:17:31.587678 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:31.587680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:31.587696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:31.587727 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:31.588800 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.589556 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.591142 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.591462 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.591508 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.594195 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:31.594196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:31.594197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:31.594205 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:31.595994 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.596534 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.596536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.596536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.596599 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.596889 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.596890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.596890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.597033 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:31.597075 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:31.597474 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 05:17:31.597572 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 05:17:31.598479 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.586236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.130:34035)
0925 05:17:31.586779 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:31.586827 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:31.587549 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:31.587923 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:31.587929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:31.588376 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:31.588691 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.588698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.589682 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.589683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.591035 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.591038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.591615 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.591616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.591755 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.592434 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:31.592444 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:31.594072 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:31.596107 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.596115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.596118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.596444 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.596668 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.596669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.596669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.596815 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.597123 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:31.597130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:31.597309 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 05:17:31.598044 (+ 735us) client_negotiation.cc:240] Negotiation successful
0925 05:17:31.598223 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":92,"threads_started":1}
I20260925 05:17:31.599220 26518 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:31.599320 26518 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:31.599524 26518 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:31.599972 26168 ts_manager.cc:194] Registered new tserver with Master: 75b8cdf1578142bf927e8d88c6dd59d0 (127.13.160.130:43333)
I20260925 05:17:31.600458 26168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:34035
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:17:31.635564 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:31.764303 26523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:31.764693 26523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:31.764788 26523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:31.766809 26523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:31.767127 26523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:31.767233 26523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:31.769672 26523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:31.770215 26523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:31.770566 26523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:31.777266 26529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:31.777714 26531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:31.778621 26523 server_base.cc:1034] running on GCE node
W20260925 05:17:31.779162 26528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:31.782402 26523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:31.782732 26523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:31.783914 26523 hybrid_clock.cc:648] HybridClock initialized: now 1790313451783898 us; error 50 us; skew 500 ppm
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:31.787160 26523 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:31.787645 26523 webserver.cc:460] Webserver started at http://127.13.160.131:44593/ using document root <none> and password file <none>
I20260925 05:17:31.787918 26523 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:31.787992 26523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:31.788144 26523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:31.801841 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "041ecb51df774fee8ebfa5affde2641c"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "59eb14d27a299a15531826602fdfc236"
server_key_iv: "09f00c1f132213a3ac7136f39a717321"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.802286 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "041ecb51df774fee8ebfa5affde2641c"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "59eb14d27a299a15531826602fdfc236"
server_key_iv: "09f00c1f132213a3ac7136f39a717321"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.804524 26523 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260925 05:17:31.837289 26538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.838819 26523 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260925 05:17:31.838972 26523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "041ecb51df774fee8ebfa5affde2641c"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "59eb14d27a299a15531826602fdfc236"
server_key_iv: "09f00c1f132213a3ac7136f39a717321"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:31.839089 26523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:31.869115 26523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:31.871869 26523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:31.872097 26523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:31.872419 26523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:31.872872 26523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:31.872963 26523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.873036 26523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:31.873096 26523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:31.881196 26523 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42267
I20260925 05:17:31.881764 26523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:17:31.884078 26651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42267 every 8 connection(s)
I20260925 05:17:31.885241 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26523
I20260925 05:17:31.885339 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:17:31.885551 13954 external_mini_cluster.cc:1428] Setting key 73c13ef85003b03f79320c4a05f5e81c
Sep 25 05:17:31 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313451, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:31.898988 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.886643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:58779 (local address 127.13.160.190:40291)
0925 05:17:31.886714 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:31.886717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:31.887453 (+ 736us) server_negotiation.cc:407] Connection header received
0925 05:17:31.888187 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:31.888188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:31.888202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:31.888233 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:31.889127 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.889909 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.891461 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.891767 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.891815 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.894650 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:31.894652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:31.894652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:31.894661 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:31.896487 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.897025 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.897027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.897027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.897093 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:31.898099 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:31.898100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:31.898101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:31.898265 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:31.898312 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:31.898746 (+ 434us) server_negotiation.cc:299] Negotiation successful
0925 05:17:31.898838 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:17:31.899930 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:31.886311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.131:58779)
0925 05:17:31.887320 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:31.887372 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:31.888060 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:31.888439 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:31.888443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:31.888722 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:31.889017 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.889025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.890029 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.890030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.891354 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:31.891357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:31.891928 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:31.891929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:31.892083 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:31.892737 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:31.892748 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:31.894509 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:31.896603 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.896611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.896614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.896932 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.897171 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:31.897172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:31.897172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:31.898010 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:31.898363 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:31.898369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:31.898566 (+ 197us) client_negotiation.cc:769] Sending connection context
0925 05:17:31.899471 (+ 905us) client_negotiation.cc:240] Negotiation successful
0925 05:17:31.899671 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":110,"threads_started":1}
I20260925 05:17:31.900656 26652 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:31.900755 26652 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:31.900955 26652 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:31.901376 26168 ts_manager.cc:194] Registered new tserver with Master: 041ecb51df774fee8ebfa5affde2641c (127.13.160.131:42267)
I20260925 05:17:31.901924 26168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:58779
I20260925 05:17:31.908722 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:17:31.909574 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26249
I20260925 05:17:31.915925 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26386
I20260925 05:17:31.921816 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26523
I20260925 05:17:31.928682 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26136
I20260925 05:17:31.934763 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40291
--webserver_interface=127.13.160.190
--webserver_port=40863
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40291
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:32.019800 26655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:32.020056 26655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:32.020095 26655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:32.021309 26655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:32.021382 26655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:32.021404 26655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:32.021423 26655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:32.021456 26655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:32.023171 26655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40291
--ranger_config_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:32.023469 26655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:32.023720 26655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:32.028825 26660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.028944 26655 server_base.cc:1034] running on GCE node
W20260925 05:17:32.028846 26661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:32.028846 26663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.029388 26655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:32.029752 26655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:32.030952 26655 hybrid_clock.cc:648] HybridClock initialized: now 1790313452030936 us; error 31 us; skew 500 ppm
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:32.033540 26655 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:32.033993 26655 webserver.cc:460] Webserver started at http://127.13.160.190:40863/ using document root <none> and password file <none>
I20260925 05:17:32.034250 26655 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:32.034325 26655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:32.100302 26655 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:32.101174 26671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.101375 26655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:32.101497 26655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "7d5ad27cfc0047f082d81e21956afe7f"
format_stamp: "Formatted at 2026-09-25 05:17:30 on dist-test-slave-9tgz"
server_key: "18d340d6f024ad8eb654f4041a2f8a11"
server_key_iv: "c69469d67c1bee35840a1f097543a164"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:32.101974 26655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:32.132146 26655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:32.133900 26655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:32.134076 26655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:32.138085 26655 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40291
I20260925 05:17:32.138180 26723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40291 every 8 connection(s)
I20260925 05:17:32.138558 26655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:32.139839 26724 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:17:32.140503 26724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Bootstrap starting.
I20260925 05:17:32.141238 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26655
I20260925 05:17:32.141637 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:40547
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:40291
--webserver_port=32823
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:32.149528 26724 log.cc:824] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:32.151486 26724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:32.151798 26724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Bootstrap complete.
I20260925 05:17:32.154258 26724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:32.154541 26724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5ad27cfc0047f082d81e21956afe7f, State: Initialized, Role: FOLLOWER
I20260925 05:17:32.154769 26724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:32.154870 26724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:32.154950 26724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:32.155005 26724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:17:32.156176 26724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:32.156306 26724 leader_election.cc:302] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d5ad27cfc0047f082d81e21956afe7f; no voters:
I20260925 05:17:32.156514 26724 leader_election.cc:288] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 05:17:32.156563 26728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 2 FOLLOWER]: Leader election won for term 2
I20260925 05:17:32.156684 26728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [term 2 LEADER]: Becoming Leader. State: Replica: 7d5ad27cfc0047f082d81e21956afe7f, State: Running, Role: LEADER
I20260925 05:17:32.156803 26724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:32.156854 26728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } }
I20260925 05:17:32.157249 26730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d5ad27cfc0047f082d81e21956afe7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } } }
I20260925 05:17:32.157246 26728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d5ad27cfc0047f082d81e21956afe7f. Latest consensus state: current_term: 2 leader_uuid: "7d5ad27cfc0047f082d81e21956afe7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5ad27cfc0047f082d81e21956afe7f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40291 } } }
I20260925 05:17:32.157356 26730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:32.157379 26728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:32.157595 26737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:32.158205 26737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:32.158401 26737 catalog_manager.cc:1259] Loaded cluster ID: 3e096dadfb95431996af3eab2099e2a6
I20260925 05:17:32.158468 26737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:17:32.158565 26724 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 05:17:32.162560 26737 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:32.163179 26737 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Loaded TSK: 0
I20260925 05:17:32.163530 26737 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:17:32.256018 26726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:32.256446 26726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:32.256515 26726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:32.258343 26726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:32.258441 26726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:32.258504 26726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:32.260852 26726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:40547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=32823
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:32.261114 26726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:32.261425 26726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:32.268143 26762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.269183 26726 server_base.cc:1034] running on GCE node
W20260925 05:17:32.269496 26761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:32.271008 26764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.271426 26726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:32.271831 26726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:32.273069 26726 hybrid_clock.cc:648] HybridClock initialized: now 1790313452273026 us; error 58 us; skew 500 ppm
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:32.276664 26726 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:32.277289 26726 webserver.cc:460] Webserver started at http://127.13.160.129:32823/ using document root <none> and password file <none>
I20260925 05:17:32.277554 26726 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:32.277621 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:32.304276 26726 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260925 05:17:32.306566 26772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.306820 26726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260925 05:17:32.306936 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "479ef5b41bf44cb8a19afdbb559ee7e3"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "3be0f6f24fb0175dbeec6eb1108545ba"
server_key_iv: "20cd88f6b1a6d774c15c8092d5b2c050"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:32.307439 26726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:32.360548 26726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:32.363217 26726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:32.363385 26726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:32.363641 26726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:32.364049 26726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:32.364104 26726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.364137 26726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:32.364198 26726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.373984 26726 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40547
I20260925 05:17:32.374611 26726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:17:32.376048 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26726
I20260925 05:17:32.376391 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:43333
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:40291
--webserver_port=45843
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:32.376371 26885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40547 every 8 connection(s)
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:32.397022 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.375656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.129:33195)
0925 05:17:32.376012 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:32.376048 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:32.376604 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:32.377110 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:32.377114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:32.377380 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:32.377735 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.377742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.379555 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.379557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.380850 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.380854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.381851 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.381852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.381985 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.382644 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:32.382655 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:32.392143 (+ 9488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:32.394834 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.394847 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.394850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.395243 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.395581 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.395582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.395582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.395731 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.396292 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:32.396299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:32.396467 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 05:17:32.396582 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:17:32.396770 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":83,"threads_started":1}
I20260925 05:17:32.397853 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.375696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33195 (local address 127.13.160.190:40291)
0925 05:17:32.375923 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:32.375934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:32.376170 (+ 236us) server_negotiation.cc:407] Connection header received
0925 05:17:32.376857 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:32.376862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:32.376909 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:32.377073 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:32.377886 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.379406 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.380962 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.381617 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.381707 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.392354 (+ 10647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:32.392361 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:32.392362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:32.392379 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:32.394691 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.395357 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.395363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.395363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.395495 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.395812 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.395813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.395813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.396173 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:32.396261 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:32.397505 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0925 05:17:32.397644 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260925 05:17:32.399144 26886 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:32.399317 26886 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.399572 26886 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.400534 26886 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:40291 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 05:17:32.400691 26886 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.400806 26886 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:32.401171 26886 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.401297 26886 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.401620 26886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 05:17:32.486336 26890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:32.486637 26890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:32.486685 26890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:32.488250 26890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:32.488329 26890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:32.488379 26890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:32.490213 26890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:43333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=45843
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:32.490530 26890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:32.490820 26890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:32.496119 26895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:32.496131 26896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:32.496577 26898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.497242 26890 server_base.cc:1034] running on GCE node
I20260925 05:17:32.497463 26890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:32.497725 26890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:32.498939 26890 hybrid_clock.cc:648] HybridClock initialized: now 1790313452498920 us; error 37 us; skew 500 ppm
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:32.501909 26890 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:32.502468 26890 webserver.cc:460] Webserver started at http://127.13.160.130:45843/ using document root <none> and password file <none>
I20260925 05:17:32.502714 26890 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:32.502795 26890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:32.518949 26890 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260925 05:17:32.523315 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.523526 26890 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 05:17:32.523643 26890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "75b8cdf1578142bf927e8d88c6dd59d0"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "f37f717c4c55690c0b8bc0a58dd3d819"
server_key_iv: "245eb8820f9eafb4bd7b5feabaa0ad34"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:32.524073 26890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:32.536412 26890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:32.539162 26890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:32.539336 26890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:32.539666 26890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:32.540102 26890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:32.540189 26890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.540239 26890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:32.540264 26890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.548014 26890 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:43333
I20260925 05:17:32.548081 27019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:43333 every 8 connection(s)
I20260925 05:17:32.548590 26890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:17:32.552093 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 26890
I20260925 05:17:32.552415 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.160.148:38539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:42267
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:40291
--webserver_port=44593
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:46033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:32.567332 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.549501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.130:42483)
0925 05:17:32.549668 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:32.549706 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:32.550217 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:32.550476 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:32.550480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:32.550666 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:32.550866 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.550869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.551662 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.551663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.552672 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.552675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.553102 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.553104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.553224 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.553816 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:32.553837 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:32.563300 (+ 9463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:32.565507 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.565521 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.565524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.565914 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.566137 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.566138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.566138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.566281 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.566576 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:32.566583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:32.566769 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 05:17:32.566878 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 05:17:32.567075 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260925 05:17:32.567869 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.549489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:42483 (local address 127.13.160.190:40291)
0925 05:17:32.549532 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:32.549535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:32.549794 (+ 259us) server_negotiation.cc:407] Connection header received
0925 05:17:32.550330 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:32.550331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:32.550347 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:32.550399 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:32.551031 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.551542 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.552763 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.552972 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.553062 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.563457 (+ 10395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:32.563460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:32.563460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:32.563472 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:32.565384 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.566007 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.566009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.566009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.566071 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.566343 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.566344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.566344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.566483 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:32.566540 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:32.567685 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0925 05:17:32.567776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:17:32.568696 27020 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:32.568799 27020 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.569005 27020 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.569669 27020 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:40291 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 05:17:32.569733 27020 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.569788 27020 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:32.570231 27020 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.570312 27020 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.571015 27020 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 05:17:32.657271 27023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:32.657634 27023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:32.657707 27023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:32.659641 27023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:32.659758 27023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:32.659837 27023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:32.662245 27023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.160.148:38539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:42267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=44593
--tserver_master_addrs=127.13.160.190:40291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:32.662528 27023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:32.662919 27023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:32.670082 27029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:32.670076 27032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.670368 27023 server_base.cc:1034] running on GCE node
W20260925 05:17:32.671049 27030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:32.671309 27023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:32.671597 27023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:32.672780 27023 hybrid_clock.cc:648] HybridClock initialized: now 1790313452672756 us; error 40 us; skew 500 ppm
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:32.675786 27023 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:32.676384 27023 webserver.cc:460] Webserver started at http://127.13.160.131:44593/ using document root <none> and password file <none>
I20260925 05:17:32.676657 27023 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:32.676749 27023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:32.693482 27023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:32.694463 27040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.694675 27023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:32.694818 27023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "041ecb51df774fee8ebfa5affde2641c"
format_stamp: "Formatted at 2026-09-25 05:17:31 on dist-test-slave-9tgz"
server_key: "59eb14d27a299a15531826602fdfc236"
server_key_iv: "09f00c1f132213a3ac7136f39a717321"
server_key_version: "kuduclusterkey@0"
I20260925 05:17:32.695324 27023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:32.704711 27023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:32.707233 27023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:32.707466 27023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:32.707800 27023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:32.708190 27023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:32.708245 27023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.708312 27023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:32.708357 27023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:32.715852 27023 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42267
I20260925 05:17:32.716500 27023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:17:32.720577 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27023
I20260925 05:17:32.721560 27153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42267 every 8 connection(s)
Sep 25 05:17:32 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313452, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:32.729338 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.721083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.13.160.190:40291)
0925 05:17:32.721143 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:32.721145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:32.721693 (+ 548us) server_negotiation.cc:407] Connection header received
0925 05:17:32.721771 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:32.721773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:32.721786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:32.721876 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:32.722989 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.723572 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.724425 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.724733 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.724806 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.726402 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:32.726404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:32.726404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:32.726411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:32.728080 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.728461 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.728462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.728462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.728545 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.728772 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.728773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.728774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.728900 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:32.729068 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:32.729160 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 05:17:32.729241 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 05:17:32.733546 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.722329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40291 (local address 127.13.160.131:54843)
0925 05:17:32.722522 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:32.722572 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:32.723276 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:32.723572 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:32.723578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:32.723798 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:32.724093 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.724101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.724875 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.724876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.725879 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:32.725882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.726311 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.726313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:32.726429 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.726995 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:32.727012 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:32.728347 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:32.731920 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.731930 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.731933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.732262 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.732486 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:32.732487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:32.732487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:32.732617 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:32.732888 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:32.732894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:32.733087 (+ 193us) client_negotiation.cc:769] Sending connection context
0925 05:17:32.733204 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 05:17:32.733365 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260925 05:17:32.733904 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:32.722660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:54843 (local address 127.13.160.190:40291)
0925 05:17:32.722815 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:32.722817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:32.722827 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:32.723397 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:32.723397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:32.723409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:32.723480 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:32.724204 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.724754 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.725969 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:32.726194 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:32.726280 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:32.728465 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:32.728466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:32.728466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:32.728472 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:32.731821 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.732349 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.732350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.732351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.732409 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:32.732700 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:32.732701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:32.732701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:32.732821 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:32.732897 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:32.733729 (+ 832us) server_negotiation.cc:299] Negotiation successful
0925 05:17:32.733810 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":55,"threads_started":1}
I20260925 05:17:32.734303 27154 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40291
I20260925 05:17:32.734416 27154 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.734625 27154 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.735399 27154 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:40291 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 05:17:32.735476 27154 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.735538 27154 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:32.735857 27154 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:32.735965 27154 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
W20260925 05:17:32.736269 27154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 05:17:33 dist-test-slave-9tgz krb5kdc[25692](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790313453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:33.401865 26886 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:33.402070 26886 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:33.402583 26688 ts_manager.cc:194] Registered new tserver with Master: 479ef5b41bf44cb8a19afdbb559ee7e3 (127.13.160.129:40547)
I20260925 05:17:33.403622 26688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:33195
I20260925 05:17:33.571209 27020 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:33.571396 27020 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:33.571776 26688 ts_manager.cc:194] Registered new tserver with Master: 75b8cdf1578142bf927e8d88c6dd59d0 (127.13.160.130:43333)
I20260925 05:17:33.572192 26688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:42483
Sep 25 05:17:33 dist-test-slave-9tgz krb5kdc[25692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313453, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for HTTP/127.13.160.148@KRBTEST.COM
I20260925 05:17:33.736479 27154 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:33.736645 27154 heartbeater.cc:511] Master 127.13.160.190:40291 requested a full tablet report, sending...
I20260925 05:17:33.737028 26688 ts_manager.cc:194] Registered new tserver with Master: 041ecb51df774fee8ebfa5affde2641c (127.13.160.131:42267)
I20260925 05:17:33.737558 26688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:54843
I20260925 05:17:33.741271 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:17:33.750626 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.742399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.13.160.190:40291)
0925 05:17:33.742583 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.742585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.742693 (+ 108us) server_negotiation.cc:407] Connection header received
0925 05:17:33.742786 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.742787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.742800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.742842 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:33.743620 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.744424 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.745483 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.745819 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.745863 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.747487 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:33.747489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:33.747489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:33.747497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:33.749358 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:33.749768 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:33.749769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:33.749769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:33.749827 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:33.750020 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:33.750021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:33.750021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:33.750151 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:33.750224 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.750433 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.750517 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260925 05:17:33.753396 26688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:17:33.755743 26688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:17:33.776402 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.769463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:43333 (local address 127.0.0.1:50960)
0925 05:17:33.770203 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.770246 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:33.770314 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:33.772312 (+ 1998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:33.772313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:33.772315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:33.772343 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.772349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.773584 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.773585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.774962 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.774965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.776063 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.776064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.776150 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.776165 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 05:17:33.776236 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 05:17:33.776296 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":92,"threads_started":1}
I20260925 05:17:33.776721 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.769808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.13.160.131:42267)
0925 05:17:33.771804 (+ 1996us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.771809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.771957 (+ 148us) server_negotiation.cc:407] Connection header received
0925 05:17:33.772044 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.772047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.772063 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.772143 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:33.772427 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.773790 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.775070 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.776179 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.776247 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.776293 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.776616 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.776656 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1961,"thread_start_us":78,"threads_started":1}
I20260925 05:17:33.777066 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.769563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50960 (local address 127.13.160.130:43333)
0925 05:17:33.772092 (+ 2529us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.772096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.772106 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:33.772138 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.772140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.772156 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.772224 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:33.772426 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.773498 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.775071 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.775890 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.775964 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.776024 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.776941 (+ 917us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.776985 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2489,"thread_start_us":48,"threads_started":1}
I20260925 05:17:33.776547 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.769708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42267 (local address 127.0.0.1:47026)
0925 05:17:33.771859 (+ 2151us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.771869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:33.771960 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:33.772261 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:33.772265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:33.772268 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:33.772343 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.772346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.773876 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.773877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.774970 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.774972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.776325 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.776325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.776384 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.776387 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:17:33.776435 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 05:17:33.776475 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2137,"thread_start_us":61,"threads_started":1}
I20260925 05:17:33.777918 26954 tablet_service.cc:1467] Processing CreateTablet for tablet f84c917c26094b438499d19b02e93658 (DEFAULT_TABLE table=test-table [id=5d91ce4c706245b991441cc2cafc64c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:33.778322 26954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c917c26094b438499d19b02e93658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:33.781610 27187 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Bootstrap starting.
I20260925 05:17:33.782505 27070 tablet_service.cc:1467] Processing CreateTablet for tablet f84c917c26094b438499d19b02e93658 (DEFAULT_TABLE table=test-table [id=5d91ce4c706245b991441cc2cafc64c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:33.782725 27187 tablet_bootstrap.cc:654] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:33.782864 27070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c917c26094b438499d19b02e93658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:33.783084 27187 log.cc:824] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:33.785859 27189 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Bootstrap starting.
I20260925 05:17:33.786516 27189 tablet_bootstrap.cc:654] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:33.786880 27189 log.cc:824] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:33.788017 27189 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: No bootstrap required, opened a new log
I20260925 05:17:33.788085 27189 ts_tablet_manager.cc:1397] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:17:33.790195 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.769858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40547 (local address 127.0.0.1:58208)
0925 05:17:33.776798 (+ 6940us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.776812 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:33.776863 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:33.779218 (+ 2355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:33.779219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:33.779221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:33.779261 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.779263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.787660 (+ 8397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.787662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.788786 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.788791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.789919 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.789921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.790012 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.790016 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:17:33.790049 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 05:17:33.790110 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6924,"thread_start_us":55,"threads_started":1}
I20260925 05:17:33.790289 27189 raft_consensus.cc:348] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.790472 27189 raft_consensus.cc:374] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:33.790511 27189 raft_consensus.cc:729] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 041ecb51df774fee8ebfa5affde2641c, State: Initialized, Role: FOLLOWER
I20260925 05:17:33.790647 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.770037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.13.160.129:40547)
0925 05:17:33.770436 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.770442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.777035 (+ 6593us) server_negotiation.cc:407] Connection header received
0925 05:17:33.777084 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.777087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.777108 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.777271 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:33.786293 (+ 9022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.787532 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.788914 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.789799 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.790387 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.790457 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.790489 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.790537 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":42,"threads_started":1}
I20260925 05:17:33.790638 27189 consensus_queue.cc:260] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.790961 27189 ts_tablet_manager.cc:1428] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 05:17:33.791356 27154 heartbeater.cc:503] Master 127.13.160.190:40291 was elected leader, sending a full tablet report...
I20260925 05:17:33.791357 26820 tablet_service.cc:1467] Processing CreateTablet for tablet f84c917c26094b438499d19b02e93658 (DEFAULT_TABLE table=test-table [id=5d91ce4c706245b991441cc2cafc64c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:33.791733 26820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c917c26094b438499d19b02e93658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:33.792070 27187 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: No bootstrap required, opened a new log
I20260925 05:17:33.792146 27187 ts_tablet_manager.cc:1397] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260925 05:17:33.795323 27187 raft_consensus.cc:348] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.795512 27187 raft_consensus.cc:374] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:33.795553 27187 raft_consensus.cc:729] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b8cdf1578142bf927e8d88c6dd59d0, State: Initialized, Role: FOLLOWER
I20260925 05:17:33.795691 27187 consensus_queue.cc:260] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.795969 27187 ts_tablet_manager.cc:1428] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 05:17:33.796096 27020 heartbeater.cc:503] Master 127.13.160.190:40291 was elected leader, sending a full tablet report...
I20260925 05:17:33.797381 27193 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: Bootstrap starting.
I20260925 05:17:33.798640 27193 tablet_bootstrap.cc:654] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:33.799081 27193 log.cc:824] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:33.799882 27193 tablet_bootstrap.cc:492] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: No bootstrap required, opened a new log
W20260925 05:17:33.799926 27021 tablet.cc:2369] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:17:33.800014 27193 ts_tablet_manager.cc:1397] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 05:17:33.802241 27193 raft_consensus.cc:348] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.802430 27193 raft_consensus.cc:374] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:33.802495 27193 raft_consensus.cc:729] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479ef5b41bf44cb8a19afdbb559ee7e3, State: Initialized, Role: FOLLOWER
I20260925 05:17:33.802652 27193 consensus_queue.cc:260] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.803000 27193 ts_tablet_manager.cc:1428] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:17:33.803051 26886 heartbeater.cc:503] Master 127.13.160.190:40291 was elected leader, sending a full tablet report...
I20260925 05:17:33.818567 27192 raft_consensus.cc:482] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:33.818746 27192 raft_consensus.cc:504] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.819195 27192 leader_election.cc:288] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 479ef5b41bf44cb8a19afdbb559ee7e3 (127.13.160.129:40547), 041ecb51df774fee8ebfa5affde2641c (127.13.160.131:42267)
I20260925 05:17:33.823558 27184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.819642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:46713 (local address 127.13.160.131:42267)
0925 05:17:33.819685 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.819688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.820013 (+ 325us) server_negotiation.cc:407] Connection header received
0925 05:17:33.820103 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.820104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.820113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.820141 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:33.820338 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.821070 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.822501 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.823093 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.823142 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.823151 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.823430 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.823480 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:17:33.823719 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.819628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39571 (local address 127.13.160.129:40547)
0925 05:17:33.819671 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.819673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.819869 (+ 196us) server_negotiation.cc:407] Connection header received
0925 05:17:33.819927 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.819928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.819938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.819998 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:33.820216 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.820973 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.822514 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.823105 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.823160 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.823169 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.823587 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.823638 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:17:33.823992 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.819389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40547 (local address 127.13.160.130:39571)
0925 05:17:33.819792 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.819804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:33.819870 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:33.820054 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:33.820055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:33.820057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:33.820139 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.820141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.821085 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.821086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.822385 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.822389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.823396 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.823397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.823493 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.823498 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:17:33.823841 (+ 343us) client_negotiation.cc:240] Negotiation successful
0925 05:17:33.823891 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":25,"thread_start_us":132,"threads_started":1}
I20260925 05:17:33.824101 27197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.819403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42267 (local address 127.13.160.130:46713)
0925 05:17:33.819966 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.819973 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:33.820035 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:33.820202 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:33.820203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:33.820204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:33.820271 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.820272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.821167 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.821168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.822380 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:33.822383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.823270 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.823271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:33.823340 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.823344 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:17:33.823973 (+ 629us) client_negotiation.cc:240] Negotiation successful
0925 05:17:33.824019 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":118,"threads_started":1}
I20260925 05:17:33.824514 26840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c917c26094b438499d19b02e93658" candidate_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" is_pre_election: true
I20260925 05:17:33.824704 26840 raft_consensus.cc:2454] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b8cdf1578142bf927e8d88c6dd59d0 in term 0.
I20260925 05:17:33.824700 27102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c917c26094b438499d19b02e93658" candidate_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "041ecb51df774fee8ebfa5affde2641c" is_pre_election: true
I20260925 05:17:33.824846 27102 raft_consensus.cc:2454] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b8cdf1578142bf927e8d88c6dd59d0 in term 0.
I20260925 05:17:33.825058 26908 leader_election.cc:302] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 041ecb51df774fee8ebfa5affde2641c, 75b8cdf1578142bf927e8d88c6dd59d0; no voters:
I20260925 05:17:33.825239 27192 raft_consensus.cc:2781] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:17:33.825278 27192 raft_consensus.cc:482] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:17:33.825299 27192 raft_consensus.cc:3037] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:33.826197 27192 raft_consensus.cc:504] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.826351 27192 leader_election.cc:288] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [CANDIDATE]: Term 1 election: Requested vote from peers 479ef5b41bf44cb8a19afdbb559ee7e3 (127.13.160.129:40547), 041ecb51df774fee8ebfa5affde2641c (127.13.160.131:42267)
I20260925 05:17:33.826530 26840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c917c26094b438499d19b02e93658" candidate_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3"
I20260925 05:17:33.826615 26840 raft_consensus.cc:3037] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:33.826684 27102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c917c26094b438499d19b02e93658" candidate_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "041ecb51df774fee8ebfa5affde2641c"
I20260925 05:17:33.826761 27102 raft_consensus.cc:3037] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:33.827605 26840 raft_consensus.cc:2454] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b8cdf1578142bf927e8d88c6dd59d0 in term 1.
I20260925 05:17:33.827625 27102 raft_consensus.cc:2454] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b8cdf1578142bf927e8d88c6dd59d0 in term 1.
I20260925 05:17:33.827826 26908 leader_election.cc:302] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 041ecb51df774fee8ebfa5affde2641c, 75b8cdf1578142bf927e8d88c6dd59d0; no voters:
I20260925 05:17:33.828130 27192 raft_consensus.cc:2781] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:33.828289 27192 raft_consensus.cc:686] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 1 LEADER]: Becoming Leader. State: Replica: 75b8cdf1578142bf927e8d88c6dd59d0, State: Running, Role: LEADER
I20260925 05:17:33.828380 27192 consensus_queue.cc:237] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } }
I20260925 05:17:33.829213 26688 catalog_manager.cc:5495] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b8cdf1578142bf927e8d88c6dd59d0 (127.13.160.130). New cstate: current_term: 1 leader_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b8cdf1578142bf927e8d88c6dd59d0" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 43333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 } health_report { overall_health: UNKNOWN } } }
I20260925 05:17:33.846114 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.842792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.13.160.130:43333)
0925 05:17:33.842836 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:33.842838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:33.842848 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:33.842954 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:33.842955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:33.842980 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:33.843063 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:17:33.843221 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.843982 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.845125 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:33.845439 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:33.845500 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:33.845708 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:17:33.845850 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:17:33.845937 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:33.846005 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 05:17:33.846029 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:17:33.848516 27102 raft_consensus.cc:1261] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 1 FOLLOWER]: Refusing update from remote peer 75b8cdf1578142bf927e8d88c6dd59d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:33.848645 26840 raft_consensus.cc:1261] T f84c917c26094b438499d19b02e93658 P 479ef5b41bf44cb8a19afdbb559ee7e3 [term 1 FOLLOWER]: Refusing update from remote peer 75b8cdf1578142bf927e8d88c6dd59d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:33.849071 27198 consensus_queue.cc:1035] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "479ef5b41bf44cb8a19afdbb559ee7e3" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:33.849241 27192 consensus_queue.cc:1035] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "041ecb51df774fee8ebfa5affde2641c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:33.853284 27201 mvcc.cc:205] Tried to move back new op lower bound from 7333123906960584704 to 7333123906881634304. Current Snapshot: MvccSnapshot[applied={T|T < 7333123906960584704}]
I20260925 05:17:33.855593 27202 mvcc.cc:205] Tried to move back new op lower bound from 7333123906960584704 to 7333123906881634304. Current Snapshot: MvccSnapshot[applied={T|T < 7333123906960584704}]
I20260925 05:17:33.855893 27204 mvcc.cc:205] Tried to move back new op lower bound from 7333123906960584704 to 7333123906881634304. Current Snapshot: MvccSnapshot[applied={T|T < 7333123906960584704}]
I20260925 05:17:33.856753 26688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:33.856849 26688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:33.857614 26688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d5ad27cfc0047f082d81e21956afe7f: Sending DeleteTablet for 3 replicas of tablet f84c917c26094b438499d19b02e93658
I20260925 05:17:33.857972 26954 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c917c26094b438499d19b02e93658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:33 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:50960
I20260925 05:17:33.858004 27070 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c917c26094b438499d19b02e93658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:33 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:47026
I20260925 05:17:33.858397 26820 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c917c26094b438499d19b02e93658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:33 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:58208
I20260925 05:17:33.858865 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26726
I20260925 05:17:33.866828 27210 tablet_replica.cc:330] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0: stopping tablet replica
I20260925 05:17:33.867034 27211 tablet_replica.cc:330] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: stopping tablet replica
I20260925 05:17:33.867085 27210 raft_consensus.cc:2229] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:17:33.867238 27210 raft_consensus.cc:2258] T f84c917c26094b438499d19b02e93658 P 75b8cdf1578142bf927e8d88c6dd59d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:33.867357 27211 raft_consensus.cc:2229] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 05:17:33.867492 26672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:40547 (error 108)
I20260925 05:17:33.867548 27211 raft_consensus.cc:2258] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:33.868115 27211 ts_tablet_manager.cc:1905] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:33.868219 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26890
I20260925 05:17:33.871381 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.871245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40547 (local address 127.0.0.1:58210)
0925 05:17:33.871290 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.871346 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 05:17:33.871250 27211 ts_tablet_manager.cc:1918] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:33.871537 27211 log.cc:1196] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/f84c917c26094b438499d19b02e93658
W20260925 05:17:33.871527 26672 catalog_manager.cc:4584] TS 479ef5b41bf44cb8a19afdbb559ee7e3 (127.13.160.129:40547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f84c917c26094b438499d19b02e93658: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40547: connect: Connection refused (error 111)
I20260925 05:17:33.871789 27211 ts_tablet_manager.cc:1939] T f84c917c26094b438499d19b02e93658 P 041ecb51df774fee8ebfa5affde2641c: Deleting consensus metadata
I20260925 05:17:33.872210 26673 catalog_manager.cc:4841] TS 041ecb51df774fee8ebfa5affde2641c (127.13.160.131:42267): tablet f84c917c26094b438499d19b02e93658 (table test-table [id=5d91ce4c706245b991441cc2cafc64c3]) successfully deleted
I20260925 05:17:33.875098 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27023
I20260925 05:17:33.875286 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:33.875155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:43333 (local address 127.0.0.1:50976)
0925 05:17:33.875202 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:33.875240 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:43333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 05:17:33.882941 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 26655
I20260925 05:17:33.893150 13954 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 05:17:34.228451 13954 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 05:17:34.228576 13954 mini_ranger.cc:66] Stopping Ranger...
I20260925 05:17:34.573465 13954 mini_ranger.cc:68] Stopped Ranger
2026-09-25 05:17:34.573 UTC [25722] LOG: received smart shutdown request
2026-09-25 05:17:34.577 UTC [25722] LOG: background worker "logical replication launcher" (PID 25731) exited with exit code 1
2026-09-25 05:17:34.578 UTC [25726] LOG: shutting down
2026-09-25 05:17:34.743 UTC [25722] LOG: database system is shut down
2026-09-25T05:17:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48168 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:17:34 dist-test-slave-9tgz krb5kdc[27222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:17:34 dist-test-slave-9tgz krb5kdc[27222](info): set up 2 sockets
Sep 25 05:17:34 dist-test-slave-9tgz krb5kdc[27222](info): commencing operation
krb5kdc: starting...
W20260925 05:17:35.883694 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:17:35 dist-test-slave-9tgz krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:17:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:17:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:17:36.004187 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:46131
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:36.086768 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:36.087087 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:36.087132 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:36.088359 27235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:36.088439 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:36.088465 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:36.088500 27235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:36.088548 27235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:36.090186 27235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:46131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:36.090497 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:36.090745 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:36.095773 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:36.095804 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:36.095779 27240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:36.096101 27235 server_base.cc:1034] running on GCE node
I20260925 05:17:36.096392 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:36.096706 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:36.097882 27235 hybrid_clock.cc:648] HybridClock initialized: now 1790313456097857 us; error 41 us; skew 500 ppm
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:36.104748 27235 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:36.105242 27235 webserver.cc:460] Webserver started at http://127.13.160.190:44775/ using document root <none> and password file <none>
I20260925 05:17:36.105479 27235 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:36.105551 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:36.105726 27235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:36.106837 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "89505523e16608557c4d2b21973c251e"
server_key_iv: "ce1ff710279e916f0e61e1fee24ef239"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.107290 27235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "89505523e16608557c4d2b21973c251e"
server_key_iv: "ce1ff710279e916f0e61e1fee24ef239"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.108692 27235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:36.109483 27250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:36.109678 27235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:36.109787 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "89505523e16608557c4d2b21973c251e"
server_key_iv: "ce1ff710279e916f0e61e1fee24ef239"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.109864 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:36.142000 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:36.144675 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:36.144871 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:36.149154 27235 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:46131
I20260925 05:17:36.149207 27302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:46131 every 8 connection(s)
I20260925 05:17:36.149605 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:36.149883 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27235
I20260925 05:17:36.149992 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:17:36.150187 13954 external_mini_cluster.cc:1428] Setting key a37a7f09cb4c227f5667010bbd160f34
I20260925 05:17:36.150601 27303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:36.152930 27303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.160.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.189@KRBTEST.COM" created.
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:36.165349 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.153815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:46131 (local address 127.0.0.1:53328)
0925 05:17:36.154034 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.154093 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.154794 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.155153 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.155155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.155455 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.155614 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.155619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.156739 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.156740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.157440 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.157442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.157962 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.157963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.158026 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.158573 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.158582 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.161834 (+ 3252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.163771 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.163777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.163779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.163985 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.164246 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.164247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.164247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.164332 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.164702 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.164707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.164768 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.164877 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.165123 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":116,"thread_start_us":93,"threads_started":1}
I20260925 05:17:36.165350 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.153872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.13.160.190:46131)
0925 05:17:36.154070 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.154080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.154203 (+ 123us) server_negotiation.cc:407] Connection header received
0925 05:17:36.154996 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.154998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.155022 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.155147 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.155719 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.156643 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.157538 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.157865 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.157995 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.161932 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.161940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.161941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.161948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.163666 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.164089 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.164097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.164098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.164175 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.164397 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.164398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.164399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.164630 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.164726 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.164904 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.165085 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260925 05:17:36.166218 27303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:17:36.166644 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.166491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49454)
0925 05:17:36.166550 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.166612 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 05:17:36.166793 27253 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
I20260925 05:17:36.167282 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.167158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49468)
0925 05:17:36.167220 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.167252 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 05:17:36.167414 27303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:46855: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
I20260925 05:17:36.175827 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:46855
--webserver_interface=127.13.160.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:36.216624 27303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } attempt: 1
I20260925 05:17:36.217128 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.216987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49472)
0925 05:17:36.217067 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.217099 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 05:17:36.217478 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.217385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49486)
0925 05:17:36.217429 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.217454 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 05:17:36.217576 27303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:46855: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
W20260925 05:17:36.258261 27310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:36.258531 27310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:36.258581 27310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:36.259804 27310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:36.259877 27310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:36.259899 27310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:36.259920 27310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:36.259953 27310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:36.261456 27310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.160.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:36.261758 27310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:36.262001 27310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:36.266839 27315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:36.266839 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:36.267056 27310 server_base.cc:1034] running on GCE node
W20260925 05:17:36.267153 27316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:36.267405 27310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:36.267711 27310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:36.268868 27310 hybrid_clock.cc:648] HybridClock initialized: now 1790313456268850 us; error 33 us; skew 500 ppm
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:36.271646 27310 init.cc:377] Logged in from keytab as kudu/127.13.160.189@KRBTEST.COM (short username kudu)
I20260925 05:17:36.272120 27310 webserver.cc:460] Webserver started at http://127.13.160.189:34193/ using document root <none> and password file <none>
I20260925 05:17:36.272372 27310 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:36.272440 27310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:36.272589 27310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:36.273779 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data/instance:
uuid: "2d0bfaec669f4c23ab1c51b5759bfeba"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "06c197825b48be056addcbeb80f9d275"
server_key_iv: "dd37330588fabb2e4f23a904b936cd03"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.274171 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal/instance:
uuid: "2d0bfaec669f4c23ab1c51b5759bfeba"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "06c197825b48be056addcbeb80f9d275"
server_key_iv: "dd37330588fabb2e4f23a904b936cd03"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.275641 27310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:17:36.276489 27325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:36.276791 27310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:36.276901 27310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "2d0bfaec669f4c23ab1c51b5759bfeba"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "06c197825b48be056addcbeb80f9d275"
server_key_iv: "dd37330588fabb2e4f23a904b936cd03"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.276980 27310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:36.285735 27303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } attempt: 2
I20260925 05:17:36.286240 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.286100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49502)
0925 05:17:36.286167 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.286209 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 05:17:36.286604 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.286500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49516)
0925 05:17:36.286552 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.286578 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 05:17:36.286702 27303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:46855: Network error: Client connection negotiation failed: client connection to 127.13.160.189:46855: connect: Connection refused (error 111)
I20260925 05:17:36.295267 27310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:36.297818 27310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:36.298002 27310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:36.302243 27310 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.189:46855
I20260925 05:17:36.302299 27377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.189:46855 every 8 connection(s)
I20260925 05:17:36.302707 27310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
I20260925 05:17:36.303671 27378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:36.305270 27378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:17:36.310838 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27310
I20260925 05:17:36.311012 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-1/wal/instance
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:36.311214 13954 external_mini_cluster.cc:1428] Setting key 2cebbda87162942f40f7e1c1aad3f85f
I20260925 05:17:36.315197 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.306029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.13.160.190:46131)
0925 05:17:36.306111 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.306113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.306309 (+ 196us) server_negotiation.cc:407] Connection header received
0925 05:17:36.306771 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.306772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.306783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.306848 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.307576 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.308065 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.309018 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.309226 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.309278 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.311674 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.311676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.311676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.311684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.313472 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.314006 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.314008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.314009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.314087 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.314331 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.314332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.314332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.314492 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.314564 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.314884 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.315032 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 05:17:36.315289 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.305979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:46131 (local address 127.0.0.1:53344)
0925 05:17:36.306171 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.306215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.306688 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.306989 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.306995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.307269 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.307475 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.307480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.308148 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.308148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.308941 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.308942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.309299 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.309300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.309408 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.309993 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.310003 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.311560 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.313564 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.313571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.313573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.313929 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.314157 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.314158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.314158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.314266 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.314583 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.314591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.314729 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.314833 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.315085 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260925 05:17:36.315907 27378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.160.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.188@KRBTEST.COM" created.
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:17:36.325958 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.316227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49524)
0925 05:17:36.316479 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.316491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.316551 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.316713 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.316714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.316885 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.316956 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.316958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.317809 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.317810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.318673 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.318676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.319238 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.319239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.319314 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.319823 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.319827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.323040 (+ 3213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.324853 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.324854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.324854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.325090 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.325312 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.325313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.325313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.325365 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.325707 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.325707 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.325745 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.325795 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.325874 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260925 05:17:36.326141 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.316306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.13.160.189:46855)
0925 05:17:36.316490 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.316494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.316543 (+ 49us) server_negotiation.cc:407] Connection header received
0925 05:17:36.316612 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.316614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.316624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.316685 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.317037 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.317717 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.318774 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.319146 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.319216 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.323125 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.323133 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.323133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.323140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.324758 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.325178 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.325182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.325182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.325245 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.325426 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.325427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.325428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.325643 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.325727 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.325829 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.325959 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260925 05:17:36.326803 27378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:17:36.327185 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.327065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51356)
0925 05:17:36.327115 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.327156 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 05:17:36.327353 27328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
I20260925 05:17:36.327792 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.327707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51364)
0925 05:17:36.327755 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.327778 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 05:17:36.327911 27378 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:42957: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
I20260925 05:17:36.338233 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:42957
--webserver_interface=127.13.160.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:36.377146 27378 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } attempt: 1
I20260925 05:17:36.377714 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.377576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51372)
0925 05:17:36.377641 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.377686 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 05:17:36.377851 27303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } attempt: 3
I20260925 05:17:36.378181 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.378076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51386)
0925 05:17:36.378114 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.378148 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260925 05:17:36.378315 27378 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:42957: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:17:36.385710 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.378185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.13.160.189:46855)
0925 05:17:36.378220 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.378222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.378246 (+ 24us) server_negotiation.cc:407] Connection header received
0925 05:17:36.378304 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.378305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.378317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.378365 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.378768 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.379383 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.380432 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.380679 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.380787 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.383051 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.383053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.383053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.383061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.384583 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.385002 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.385003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.385003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.385056 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.385250 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.385251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.385251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.385358 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.385408 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.385567 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.385641 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:17:36.385766 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.378132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49538)
0925 05:17:36.378194 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.378206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.378245 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.378395 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.378397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.378588 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.378673 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.378675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.379473 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.379474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.380323 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.380326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.380799 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.380800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.380888 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.381425 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.381431 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.382961 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.384678 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.384679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.384680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.384913 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.385121 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.385122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.385122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.385190 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.385418 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.385419 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.385474 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.385551 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.385666 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 05:17:36.386267 27303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:17:36.386654 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.386534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51392)
0925 05:17:36.386588 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.386626 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 05:17:36.387048 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.386945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51404)
0925 05:17:36.386990 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.387028 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 05:17:36.387149 27303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:42957: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
I20260925 05:17:36.418331 27303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } attempt: 1
I20260925 05:17:36.418919 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.418728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51410)
0925 05:17:36.418800 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.418851 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 05:17:36.419334 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.419224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51412)
0925 05:17:36.419277 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.419311 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 05:17:36.419459 27303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:42957: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
W20260925 05:17:36.424855 27385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:36.425076 27385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:36.425128 27385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:36.426307 27385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:36.426353 27385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:36.426367 27385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:36.426380 27385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:36.426393 27385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:36.427994 27385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:46131,127.13.160.189:46855,127.13.160.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.160.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:36.428261 27385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:36.428514 27385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:36.433483 27390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:36.433624 27385 server_base.cc:1034] running on GCE node
W20260925 05:17:36.433501 27393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:36.433483 27391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:36.434016 27385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:36.434343 27385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:36.435485 27385 hybrid_clock.cc:648] HybridClock initialized: now 1790313456435470 us; error 36 us; skew 500 ppm
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:36.437952 27385 init.cc:377] Logged in from keytab as kudu/127.13.160.188@KRBTEST.COM (short username kudu)
I20260925 05:17:36.438411 27385 webserver.cc:460] Webserver started at http://127.13.160.188:44585/ using document root <none> and password file <none>
I20260925 05:17:36.438663 27385 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:36.438735 27385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:36.438884 27385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:36.440030 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data/instance:
uuid: "23e75b8a6f96425baa07ce9399362526"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "eff6adb25f79c0083a481232072045e7"
server_key_iv: "603a115285afa9e45e325e5417d92fb2"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.440413 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal/instance:
uuid: "23e75b8a6f96425baa07ce9399362526"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "eff6adb25f79c0083a481232072045e7"
server_key_iv: "603a115285afa9e45e325e5417d92fb2"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.441753 27385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:17:36.442533 27400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:36.442713 27385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:36.442818 27385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "23e75b8a6f96425baa07ce9399362526"
format_stamp: "Formatted at 2026-09-25 05:17:36 on dist-test-slave-9tgz"
server_key: "eff6adb25f79c0083a481232072045e7"
server_key_iv: "603a115285afa9e45e325e5417d92fb2"
server_key_version: "encryptionkey@0"
I20260925 05:17:36.442929 27385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:36.446470 27378 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } attempt: 2
I20260925 05:17:36.446952 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.446774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51416)
0925 05:17:36.446841 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.446914 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 05:17:36.447304 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.447214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51426)
0925 05:17:36.447260 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.447286 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 05:17:36.447427 27378 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:42957: Network error: Client connection negotiation failed: client connection to 127.13.160.188:42957: connect: Connection refused (error 111)
I20260925 05:17:36.465270 27385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:36.467363 27385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:36.467536 27385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:36.471757 27385 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.188:42957
I20260925 05:17:36.471836 27452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.188:42957 every 8 connection(s)
I20260925 05:17:36.472216 27385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
I20260925 05:17:36.473127 27453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:36.473574 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27385
I20260925 05:17:36.473696 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790313300911901-13954-0/minicluster-data/master-2/wal/instance
I20260925 05:17:36.473879 13954 external_mini_cluster.cc:1428] Setting key c5dc87987553ea22106238182d0a6fcd
I20260925 05:17:36.475436 27453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:36.485042 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.476273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53354 (local address 127.13.160.190:46131)
0925 05:17:36.476320 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.476323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.476479 (+ 156us) server_negotiation.cc:407] Connection header received
0925 05:17:36.476568 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.476569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.476579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.476625 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.477114 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.477830 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.478916 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.479216 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.479268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.482039 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.482041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.482041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.482048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.483509 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.484026 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.484027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.484028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.484111 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.484374 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.484374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.484375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.484540 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.484619 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.484846 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.484942 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:17:36.485303 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.476219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:46131 (local address 127.0.0.1:53354)
0925 05:17:36.476372 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.476418 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.476496 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.476640 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.476643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.476930 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.477020 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.477024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.477921 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.477922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.478813 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.478815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.479296 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.479297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.479379 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.479876 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.479887 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.481964 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.483629 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.483636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.483638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.483940 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.484184 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.484185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.484185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.484303 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.484616 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.484622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.484693 (+ 71us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.484808 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.485004 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260925 05:17:36.485306 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.474456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.13.160.188:42957)
0925 05:17:36.474637 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.474647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.474686 (+ 39us) server_negotiation.cc:407] Connection header received
0925 05:17:36.475187 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.475195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.475248 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.475377 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.476159 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.477132 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.478126 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.478713 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.478831 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.481459 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.481468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.481468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.481474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.483692 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.484120 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.484123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.484124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.484233 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.484438 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.484439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.484440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.484712 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.484859 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.484936 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.485086 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260925 05:17:36.486078 27453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:36.493588 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.486440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.13.160.189:46855)
0925 05:17:36.486504 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.486506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.486513 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:17:36.486536 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.486537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.486547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.486604 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.486986 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.487641 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.488582 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.488795 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.488861 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.490842 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.490844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.490845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.490854 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.492512 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.492873 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.492874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.492875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.492935 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.493108 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.493109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.493109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.493258 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.493337 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.493436 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.493510 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:17:36.493669 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.486344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:46855 (local address 127.0.0.1:49540)
0925 05:17:36.486396 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.486407 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.486448 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.486613 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.486614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.486798 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.486849 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.486851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.487727 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.487728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.488496 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.488498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.488872 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.488873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.488918 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.489347 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.489352 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.490748 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.492606 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.492607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.492608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.492798 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.493001 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.493002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.493002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.493046 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.493323 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.493324 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.493362 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.493409 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.493539 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 05:17:36.494107 27453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:17:36.494632 27303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } attempt: 2
I20260925 05:17:36.494836 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.486601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.13.160.190:46131)
0925 05:17:36.486639 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.486641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.486648 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:17:36.486665 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.486665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.486673 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.486725 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.487279 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.487889 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.488792 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.489054 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.489119 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.491762 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.491763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.491763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.491770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.493384 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.493769 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.493770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.493771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.493831 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.494053 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.494054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.494054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.494173 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.494372 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.494584 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.494660 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:17:36.502529 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.494407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51442)
0925 05:17:36.494479 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.494490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.494545 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.494690 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.494691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.494854 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.494916 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.494918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.495794 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.495795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.496610 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.496612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.497105 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.497162 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.497608 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.497613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.499082 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.501060 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.501061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.501062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.501269 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.501498 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.501499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.501499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.501573 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.502083 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.502084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.502137 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.502291 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.502402 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 05:17:36.502530 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.494462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.13.160.188:42957)
0925 05:17:36.494499 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.494500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.494530 (+ 30us) server_negotiation.cc:407] Connection header received
0925 05:17:36.494599 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.494600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.494610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.494657 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.494997 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.495718 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.496695 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.497028 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497093 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.499180 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.499181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.499182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.499187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.500983 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.501360 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.501361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.501362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.501418 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.501656 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.501657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.501658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.501781 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.502117 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.502327 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.502452 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":20}
I20260925 05:17:36.503588 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.494859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51458)
0925 05:17:36.494934 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.494944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.494977 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.495277 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.495277 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.495434 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.495478 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.495481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.496832 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.496833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.497471 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.497474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.497969 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.498040 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.498548 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.498553 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.499944 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.502388 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.502390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.502390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.502633 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.502857 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.502858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.502859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.502986 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.503266 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.503267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.503322 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.503385 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.503494 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 05:17:36.503644 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.494929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.13.160.188:42957)
0925 05:17:36.495120 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.495122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.495131 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:17:36.495153 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.495153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.495168 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.495563 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.495611 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.496743 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497579 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.497878 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.498190 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.500068 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.500070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.500070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.500077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.502281 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.502724 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.502725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.502725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.502781 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.503064 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.503066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.503066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.503199 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.503259 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.503445 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.503535 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":144,"threads_started":1}
I20260925 05:17:36.504180 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.495647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.13.160.189:46855)
0925 05:17:36.495680 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.495682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.495690 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:17:36.495737 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.495738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.495747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.495796 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.496284 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.496809 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497680 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.497888 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.497972 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.500891 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.500893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.500894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.500901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.502792 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.503189 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.503191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.503191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.503248 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.503452 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.503453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.503453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.503591 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.503767 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.503989 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.504102 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:17:36.504513 27453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: Bootstrap starting.
I20260925 05:17:36.505088 27453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:36.505110 27303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: Bootstrap starting.
I20260925 05:17:36.505311 27453 log.cc:824] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:36.505730 27303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:36.506006 27303 log.cc:824] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:36.506086 27453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: No bootstrap required, opened a new log
I20260925 05:17:36.506788 27303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: No bootstrap required, opened a new log
I20260925 05:17:36.507741 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:17:36.508245 27453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.508416 27453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:36.508455 27453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e75b8a6f96425baa07ce9399362526, State: Initialized, Role: FOLLOWER
I20260925 05:17:36.508637 27453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.508718 27303 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.508876 27303 raft_consensus.cc:374] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:36.508908 27303 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea15e7b52c24f4b8e8fbdbf995a52a6, State: Initialized, Role: FOLLOWER
I20260925 05:17:36.508991 27453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:36.509061 27303 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.509374 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.509407 27303 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:36.510536 27463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.510810 27463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:17:36.511273 27483 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:17:36.511376 27483 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:17:36.512187 27235 master_runner.cc:186] Error getting master registration for 127.13.160.189:46855: OK, instance_id { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" instance_seqno: 1790313456301151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:17:36.515514 27235 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:17:36.515619 27385 master_runner.cc:186] Error getting master registration for 127.13.160.189:46855: OK, instance_id { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" instance_seqno: 1790313456301151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:17:36.517928 27385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:17:36.517958 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.509214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.13.160.190:46131)
0925 05:17:36.509256 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.509258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.509510 (+ 252us) server_negotiation.cc:407] Connection header received
0925 05:17:36.509600 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.509601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.509613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.509644 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.511110 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.511830 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.512832 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.513141 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.513242 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.514875 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.514877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.514877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.514884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.516669 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.517095 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.517097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.517097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.517161 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.517391 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.517392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.517392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.517537 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.517637 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.517784 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.517867 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:17:36.509546 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:17:36.519222 27490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:17:36.519326 27490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:17:36.526646 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.510994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.13.160.189:46855)
0925 05:17:36.511314 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.511317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.517999 (+ 6682us) server_negotiation.cc:407] Connection header received
0925 05:17:36.518112 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.518114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.518127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.518161 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.518854 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.519604 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.520640 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.520934 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.520974 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.522482 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.522483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.522484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.522492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.525290 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.525726 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.525727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.525728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.525787 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.525997 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.525997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.525998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.526138 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.526216 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.526445 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.526536 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260925 05:17:36.527690 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.512591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.13.160.188:42957)
0925 05:17:36.512626 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.512628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.519681 (+ 7053us) server_negotiation.cc:407] Connection header received
0925 05:17:36.519803 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.519805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.519816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.519845 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.520534 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.521225 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.522313 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.522607 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.522652 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.524150 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.524152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.524152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.524161 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.525907 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.526279 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.526280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.526280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.526335 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.526534 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.526535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.526535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.526658 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.526728 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.527479 (+ 751us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.527569 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:17:36.538609 27378 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } attempt: 3
Sep 25 05:17:36 dist-test-slave-9tgz krb5kdc[27222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313456, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:17:36.545692 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.538969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:42957 (local address 127.0.0.1:51486)
0925 05:17:36.539011 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:36.539024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:36.539070 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:36.539189 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:36.539190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:36.539358 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:36.539400 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.539402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.540047 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.540050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.540547 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:36.540549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.540926 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.540927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:36.540992 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.541468 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:36.541472 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:36.542756 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:36.544568 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.544570 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.544570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.544749 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.544953 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:36.544954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:36.544955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:36.545061 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:36.545305 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:36.545306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:36.545393 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 05:17:36.545458 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 05:17:36.545578 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 05:17:36.545727 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:36.539034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.13.160.188:42957)
0925 05:17:36.539086 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:36.539087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:36.539096 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:17:36.539121 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:36.539122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:36.539131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:36.539181 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:36.539481 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.539955 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.540650 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:36.540842 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:36.540924 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:36.542841 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:36.542843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:36.542844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:36.542852 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:36.544453 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.544825 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.544826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.544827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.544886 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:36.545115 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:36.545116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:36.545116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:36.545238 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:36.545313 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:36.545498 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 05:17:36.545573 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 05:17:36.547595 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: Bootstrap starting.
I20260925 05:17:36.548275 27378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:36.548659 27378 log.cc:824] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:36.549510 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: No bootstrap required, opened a new log
I20260925 05:17:36.551862 27378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.552012 27378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:36.552078 27378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0bfaec669f4c23ab1c51b5759bfeba, State: Initialized, Role: FOLLOWER
I20260925 05:17:36.552217 27378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.552515 27495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.552605 27378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:36.552651 27495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:17:36.554556 27506 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:17:36.554673 27506 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:17:36.556643 27310 master_runner.cc:263] No leader master found from master 2d0bfaec669f4c23ab1c51b5759bfeba
I20260925 05:17:36.556690 27310 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:17:36.620304 27495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:36.620431 27495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.620661 27495 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eea15e7b52c24f4b8e8fbdbf995a52a6 (127.13.160.190:46131), 23e75b8a6f96425baa07ce9399362526 (127.13.160.188:42957)
I20260925 05:17:36.621102 27277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" is_pre_election: true
I20260925 05:17:36.621263 27277 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d0bfaec669f4c23ab1c51b5759bfeba in term 0.
I20260925 05:17:36.621639 27326 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d0bfaec669f4c23ab1c51b5759bfeba, eea15e7b52c24f4b8e8fbdbf995a52a6; no voters:
I20260925 05:17:36.621829 27495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:17:36.621877 27495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:17:36.621898 27495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:36.622790 27495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.622946 27495 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [CANDIDATE]: Term 1 election: Requested vote from peers eea15e7b52c24f4b8e8fbdbf995a52a6 (127.13.160.190:46131), 23e75b8a6f96425baa07ce9399362526 (127.13.160.188:42957)
I20260925 05:17:36.623291 27277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6"
I20260925 05:17:36.623371 27277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:36.624256 27277 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d0bfaec669f4c23ab1c51b5759bfeba in term 1.
I20260925 05:17:36.624490 27326 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d0bfaec669f4c23ab1c51b5759bfeba, eea15e7b52c24f4b8e8fbdbf995a52a6; no voters:
I20260925 05:17:36.624809 27426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23e75b8a6f96425baa07ce9399362526"
I20260925 05:17:36.624944 27426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:36.625883 27426 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d0bfaec669f4c23ab1c51b5759bfeba in term 1.
I20260925 05:17:36.626044 27427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23e75b8a6f96425baa07ce9399362526" is_pre_election: true
I20260925 05:17:36.626143 27427 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2d0bfaec669f4c23ab1c51b5759bfeba in term 1. Re-sending same reply.
I20260925 05:17:36.626552 27495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:36.626739 27495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [term 1 LEADER]: Becoming Leader. State: Replica: 2d0bfaec669f4c23ab1c51b5759bfeba, State: Running, Role: LEADER
I20260925 05:17:36.626827 27495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } }
I20260925 05:17:36.627349 27495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0bfaec669f4c23ab1c51b5759bfeba. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.627422 27495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:36.628300 27510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:36.628535 27510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:36.630177 27427 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [term 1 FOLLOWER]: Refusing update from remote peer 2d0bfaec669f4c23ab1c51b5759bfeba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:36.630628 27508 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [LEADER]: Connected to new peer: Peer: permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:36.632102 27463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0bfaec669f4c23ab1c51b5759bfeba. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.632213 27463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:17:36.633630 27277 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [term 1 FOLLOWER]: Refusing update from remote peer 2d0bfaec669f4c23ab1c51b5759bfeba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:36.633951 27508 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [LEADER]: Connected to new peer: Peer: permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:36.635474 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0bfaec669f4c23ab1c51b5759bfeba. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.635571 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:17:36.637153 27508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.637252 27508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:36.637706 27466 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.637787 27466 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:17:36.638073 27511 mvcc.cc:205] Tried to move back new op lower bound from 7333123918354558976 to 7333123918344552448. Current Snapshot: MvccSnapshot[applied={T|T < 7333123918354558976}]
I20260925 05:17:36.638504 27495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.638581 27495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:36.639027 27463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea15e7b52c24f4b8e8fbdbf995a52a6" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 46131 } } peers { permanent_uuid: "2d0bfaec669f4c23ab1c51b5759bfeba" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 46855 } } peers { permanent_uuid: "23e75b8a6f96425baa07ce9399362526" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 42957 } } }
I20260925 05:17:36.639130 27463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:17:36.639880 27512 mvcc.cc:205] Tried to move back new op lower bound from 7333123918354558976 to 7333123918344552448. Current Snapshot: MvccSnapshot[applied={T|T < 7333123918354558976}]
I20260925 05:17:36.640355 27514 mvcc.cc:205] Tried to move back new op lower bound from 7333123918354558976 to 7333123918344552448. Current Snapshot: MvccSnapshot[applied={T|T < 7333123918354558976}]
I20260925 05:17:36.640960 27510 catalog_manager.cc:1347] Generated new cluster ID: ee25958a3ec843b7826b8cac133053af
I20260925 05:17:36.641008 27510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:17:36.715026 27510 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:17:36.715901 27510 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:36.722913 27510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0bfaec669f4c23ab1c51b5759bfeba: Generated new TSK 0
I20260925 05:17:36.723248 27510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:17:37.511857 27483 catalog_manager.cc:1259] Loaded cluster ID: ee25958a3ec843b7826b8cac133053af
I20260925 05:17:37.511961 27483 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: loading cluster ID for follower catalog manager: success
I20260925 05:17:37.512876 27483 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: acquiring CA information for follower catalog manager: success
I20260925 05:17:37.513195 27483 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23e75b8a6f96425baa07ce9399362526: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:17:37.519793 27490 catalog_manager.cc:1259] Loaded cluster ID: ee25958a3ec843b7826b8cac133053af
I20260925 05:17:37.519863 27490 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: loading cluster ID for follower catalog manager: success
I20260925 05:17:37.520669 27490 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: acquiring CA information for follower catalog manager: success
I20260925 05:17:37.520972 27490 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eea15e7b52c24f4b8e8fbdbf995a52a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:17:37.791445 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27235
I20260925 05:17:37.796823 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27310
I20260925 05:17:37.801712 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27385
2026-09-25T05:17:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2949 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:17:37 dist-test-slave-9tgz krb5kdc[27524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:17:37 dist-test-slave-9tgz krb5kdc[27524](info): set up 2 sockets
Sep 25 05:17:37 dist-test-slave-9tgz krb5kdc[27524](info): commencing operation
krb5kdc: starting...
W20260925 05:17:38.824352 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:17:38 dist-test-slave-9tgz krb5kdc[27524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:17:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:17:38.945966 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40513
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:39.029687 27537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:39.029973 27537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:39.030010 27537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:39.031311 27537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:39.031380 27537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:39.031404 27537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:39.031423 27537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:39.031453 27537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:39.033095 27537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:39.033413 27537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:39.033668 27537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:39.038684 27545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:39.038702 27537 server_base.cc:1034] running on GCE node
W20260925 05:17:39.038684 27542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:39.038691 27543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:39.039216 27537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:39.039533 27537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:39.040695 27537 hybrid_clock.cc:648] HybridClock initialized: now 1790313459040677 us; error 40 us; skew 500 ppm
Sep 25 05:17:39 dist-test-slave-9tgz krb5kdc[27524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313459, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:39.043493 27537 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:39.044027 27537 webserver.cc:460] Webserver started at http://127.13.160.190:39247/ using document root <none> and password file <none>
I20260925 05:17:39.044277 27537 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:39.044353 27537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:39.044508 27537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:39.045676 27537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "460a1b6e44db4699ac055077b4635eb8"
format_stamp: "Formatted at 2026-09-25 05:17:39 on dist-test-slave-9tgz"
server_key: "f207ff28da6381767cb1523fabaa2dc5"
server_key_iv: "3b15dfaed6c446f517ba855ef9c4270a"
server_key_version: "encryptionkey@0"
I20260925 05:17:39.046063 27537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "460a1b6e44db4699ac055077b4635eb8"
format_stamp: "Formatted at 2026-09-25 05:17:39 on dist-test-slave-9tgz"
server_key: "f207ff28da6381767cb1523fabaa2dc5"
server_key_iv: "3b15dfaed6c446f517ba855ef9c4270a"
server_key_version: "encryptionkey@0"
I20260925 05:17:39.047600 27537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:17:39.048379 27552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:39.048558 27537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:39.048677 27537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "460a1b6e44db4699ac055077b4635eb8"
format_stamp: "Formatted at 2026-09-25 05:17:39 on dist-test-slave-9tgz"
server_key: "f207ff28da6381767cb1523fabaa2dc5"
server_key_iv: "3b15dfaed6c446f517ba855ef9c4270a"
server_key_version: "encryptionkey@0"
I20260925 05:17:39.048758 27537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:39.059310 27537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:39.061682 27537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:39.061856 27537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:39.066169 27537 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40513
I20260925 05:17:39.066206 27604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40513 every 8 connection(s)
I20260925 05:17:39.066596 27537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:39.067571 27605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:39.069936 27605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8: Bootstrap starting.
I20260925 05:17:39.070500 27605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:39.070736 27605 log.cc:824] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:39.071117 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27537
I20260925 05:17:39.071235 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:17:39.071432 13954 external_mini_cluster.cc:1428] Setting key d82dd502f049ab5c569b7815818007ef
I20260925 05:17:39.071554 27605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8: No bootstrap required, opened a new log
I20260925 05:17:39.073812 27605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } }
I20260925 05:17:39.074007 27605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:39.074048 27605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460a1b6e44db4699ac055077b4635eb8, State: Initialized, Role: FOLLOWER
I20260925 05:17:39.074206 27605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } }
I20260925 05:17:39.074301 27605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:39.074342 27605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:39.074396 27605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:39.075181 27605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } }
I20260925 05:17:39.075294 27605 leader_election.cc:302] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 460a1b6e44db4699ac055077b4635eb8; no voters:
I20260925 05:17:39.075456 27605 leader_election.cc:288] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:17:39.075524 27610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:39.075712 27605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:39.075718 27610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [term 1 LEADER]: Becoming Leader. State: Replica: 460a1b6e44db4699ac055077b4635eb8, State: Running, Role: LEADER
I20260925 05:17:39.075934 27610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } }
I20260925 05:17:39.076356 27612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460a1b6e44db4699ac055077b4635eb8. Latest consensus state: current_term: 1 leader_uuid: "460a1b6e44db4699ac055077b4635eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } } }
I20260925 05:17:39.076496 27612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:39.076690 27611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "460a1b6e44db4699ac055077b4635eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460a1b6e44db4699ac055077b4635eb8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40513 } } }
I20260925 05:17:39.077082 27611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:39.077224 27619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:39.077877 27619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:39.079689 27619 catalog_manager.cc:1347] Generated new cluster ID: b4d53895179d41049eff9529dc958eab
I20260925 05:17:39.079746 27619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:17:39 dist-test-slave-9tgz krb5kdc[27524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:39.084142 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:39.071964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.13.160.190:40513)
0925 05:17:39.072158 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:39.072168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:39.072207 (+ 39us) server_negotiation.cc:407] Connection header received
0925 05:17:39.072994 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:39.073004 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:39.073080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:39.073240 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:39.073837 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:39.075607 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:39.076843 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:39.077518 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:39.077603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:39.080540 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:39.080544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:39.080544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:39.080556 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:39.082322 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:39.082736 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:39.082739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:39.082739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:39.082833 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:39.083073 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:39.083074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:39.083075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:39.083328 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:39.083505 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:39.083864 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 05:17:39.083985 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260925 05:17:39.085224 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:17:39.087005 27619 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:17:39.087523 27619 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:39.098372 27619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 460a1b6e44db4699ac055077b4635eb8: Generated new TSK 0
I20260925 05:17:39.098621 27619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:17:39.139509 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27537
2026-09-25T05:17:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1336 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:17:39 dist-test-slave-9tgz krb5kdc[27632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:17:39 dist-test-slave-9tgz krb5kdc[27632](info): set up 2 sockets
Sep 25 05:17:39 dist-test-slave-9tgz krb5kdc[27632](info): commencing operation
krb5kdc: starting...
W20260925 05:17:40.171903 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:17:40.293598 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:40.377699 27645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:40.377939 27645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:40.377964 27645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:40.379264 27645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:40.379313 27645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:40.379328 27645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:40.379341 27645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:40.379395 27645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:40.381008 27645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:40.381265 27645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:40.381492 27645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:17:40.386808 27645 server_base.cc:1034] running on GCE node
W20260925 05:17:40.386833 27650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.386833 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.386956 27651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:40.387387 27645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:40.387727 27645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:40.388901 27645 hybrid_clock.cc:648] HybridClock initialized: now 1790313460388901 us; error 38 us; skew 500 ppm
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:40.391535 27645 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:40.392066 27645 webserver.cc:460] Webserver started at http://127.13.160.190:43753/ using document root <none> and password file <none>
I20260925 05:17:40.392313 27645 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:40.392385 27645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:40.392539 27645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:40.393708 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "6083a35baf8e4919b1d17843ceda322e"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "eca67cfdd9d0b326b84a9120aea6613c"
server_key_iv: "458f54edd6c38bb415690b45c1d8caf5"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.394091 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "6083a35baf8e4919b1d17843ceda322e"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "eca67cfdd9d0b326b84a9120aea6613c"
server_key_iv: "458f54edd6c38bb415690b45c1d8caf5"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.395556 27645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:17:40.396298 27660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.396471 27645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:40.396584 27645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "6083a35baf8e4919b1d17843ceda322e"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "eca67cfdd9d0b326b84a9120aea6613c"
server_key_iv: "458f54edd6c38bb415690b45c1d8caf5"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.396667 27645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:40.412173 27645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:40.414321 27645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:40.414507 27645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:40.418608 27645 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33621
I20260925 05:17:40.418640 27712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33621 every 8 connection(s)
I20260925 05:17:40.419175 27645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:40.420200 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:40.422538 27713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap starting.
I20260925 05:17:40.423229 27713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:40.423523 27713 log.cc:824] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:40.424264 27713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: No bootstrap required, opened a new log
I20260925 05:17:40.425899 27713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:40.426059 27713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:40.426082 27713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Initialized, Role: FOLLOWER
I20260925 05:17:40.426266 27713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:40.426330 27713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:40.426388 27713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:40.426442 27713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:40.427420 27713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:40.427537 27713 leader_election.cc:302] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6083a35baf8e4919b1d17843ceda322e; no voters:
I20260925 05:17:40.427731 27713 leader_election.cc:288] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:17:40.427809 27716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:40.427977 27716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 LEADER]: Becoming Leader. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Running, Role: LEADER
I20260925 05:17:40.428061 27713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:40.428117 27716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:40.428561 27718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6083a35baf8e4919b1d17843ceda322e. Latest consensus state: current_term: 1 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:40.428660 27718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:40.428880 27725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:40.428966 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27645
I20260925 05:17:40.429116 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:17:40.429314 13954 external_mini_cluster.cc:1428] Setting key c68c56d7f3fa990c9260bb0a848c4b16
I20260925 05:17:40.429677 27717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:40.429827 27717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:40.430166 27725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:40.432101 27725 catalog_manager.cc:1347] Generated new cluster ID: f57ca2ccd70d4024ad87cac2d3365b08
I20260925 05:17:40.432216 27725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:40.442122 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.429991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.13.160.190:33621)
0925 05:17:40.430353 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.430363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.430596 (+ 233us) server_negotiation.cc:407] Connection header received
0925 05:17:40.430938 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.430946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.431016 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.431170 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:40.431957 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.433387 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.434508 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.435147 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.435243 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.438135 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:40.438140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:40.438140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:40.438155 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:40.440322 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.440732 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.440736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.440737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.440861 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.441077 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.441078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.441079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.441434 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:40.441585 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.441707 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.441857 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260925 05:17:40.446774 27725 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:17:40.447393 27725 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:40.451104 27725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Generated new TSK 0
I20260925 05:17:40.451319 27725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:17:40.478073 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:40.564610 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:40.564845 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:40.564873 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:40.566202 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:40.566256 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:40.566284 27739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:40.567909 27739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:40.568131 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:40.568333 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:40.573616 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.573606 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:40.573820 27739 server_base.cc:1034] running on GCE node
W20260925 05:17:40.573598 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:40.574121 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:40.574432 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:40.575613 27739 hybrid_clock.cc:648] HybridClock initialized: now 1790313460575597 us; error 28 us; skew 500 ppm
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:40.578208 27739 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:40.578696 27739 webserver.cc:460] Webserver started at http://127.13.160.129:44405/ using document root <none> and password file <none>
I20260925 05:17:40.579020 27739 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:40.579095 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:40.579268 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:40.580390 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "82ea74039083468798a8177f69115cc8"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "5bb24669033a900966b27a6b099d5857"
server_key_iv: "e097f47139108c8b6780468a025a76de"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.580789 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "82ea74039083468798a8177f69115cc8"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "5bb24669033a900966b27a6b099d5857"
server_key_iv: "e097f47139108c8b6780468a025a76de"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.582182 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:17:40.583053 27754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.583211 27739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:40.583299 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "82ea74039083468798a8177f69115cc8"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "5bb24669033a900966b27a6b099d5857"
server_key_iv: "e097f47139108c8b6780468a025a76de"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.583356 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:40.598316 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:40.601027 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:40.601148 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:40.601352 27739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:40.601629 27739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:40.601665 27739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.601687 27739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:40.601699 27739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.607754 27739 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40971
I20260925 05:17:40.607803 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40971 every 8 connection(s)
I20260925 05:17:40.608217 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:17:40.613684 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27739
I20260925 05:17:40.613819 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:17:40.614029 13954 external_mini_cluster.cc:1428] Setting key 71986c432910ba234c98504123b7727d
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:40.618340 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.609032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:43917 (local address 127.13.160.190:33621)
0925 05:17:40.609105 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.609106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.609291 (+ 185us) server_negotiation.cc:407] Connection header received
0925 05:17:40.609794 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.609795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.609805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.609869 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:40.610849 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.611368 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.612372 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.612569 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.612620 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.615021 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:40.615023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:40.615023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:40.615032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:40.616647 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.617198 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.617199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.617199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.617253 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.617570 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.617571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.617572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.617729 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:40.617806 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.618089 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.618194 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:17:40.618350 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.608983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:43917)
0925 05:17:40.609152 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.609199 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.609708 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.609944 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.609948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.610546 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:40.610757 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.610763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.611461 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.611462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.612291 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.612294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.612646 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.612647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.612755 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.613273 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:40.613281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:40.614869 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:40.616741 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.616750 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.616753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.617121 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.617321 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.617322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.617323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.617474 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.617799 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:40.617804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:40.617934 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.618038 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.618173 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260925 05:17:40.618959 27868 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:40.619088 27868 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:40.619325 27868 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:40.619904 27677 ts_manager.cc:194] Registered new tserver with Master: 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971)
I20260925 05:17:40.620553 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:43917
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:17:40.641142 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:40.726039 27873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:40.726261 27873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:40.726289 27873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:40.727614 27873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:40.727661 27873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:40.727691 27873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:40.729429 27873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:40.729693 27873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:40.729944 27873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:40.735350 27879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.735394 27878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.735350 27881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:40.735647 27873 server_base.cc:1034] running on GCE node
I20260925 05:17:40.735869 27873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:40.736141 27873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:40.737323 27873 hybrid_clock.cc:648] HybridClock initialized: now 1790313460737312 us; error 29 us; skew 500 ppm
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:40.740010 27873 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:40.740438 27873 webserver.cc:460] Webserver started at http://127.13.160.130:39081/ using document root <none> and password file <none>
I20260925 05:17:40.740631 27873 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:40.740672 27873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:40.740764 27873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:40.741818 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "19c5e28f6e2b4b7fb720dfab80114732"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "83edcb6b10e2235d1bec0321dfc5aeda"
server_key_iv: "6b813c2c857c97a23ce26d013261e667"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.742141 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "19c5e28f6e2b4b7fb720dfab80114732"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "83edcb6b10e2235d1bec0321dfc5aeda"
server_key_iv: "6b813c2c857c97a23ce26d013261e667"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.743516 27873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 05:17:40.744251 27888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.744429 27873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:40.744545 27873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "19c5e28f6e2b4b7fb720dfab80114732"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "83edcb6b10e2235d1bec0321dfc5aeda"
server_key_iv: "6b813c2c857c97a23ce26d013261e667"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.744624 27873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:40.779726 27873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:40.782274 27873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:40.782454 27873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:40.782709 27873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:40.783121 27873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:40.783178 27873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.783222 27873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:40.783249 27873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.789399 27873 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:33817
I20260925 05:17:40.789448 28001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:33817 every 8 connection(s)
I20260925 05:17:40.789812 27873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:40.796711 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 27873
I20260925 05:17:40.796823 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:17:40.797026 13954 external_mini_cluster.cc:1428] Setting key a9c7e1413ac8097731c6290bf5ef84f0
I20260925 05:17:40.799909 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.790554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39247 (local address 127.13.160.190:33621)
0925 05:17:40.790638 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.790641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.790846 (+ 205us) server_negotiation.cc:407] Connection header received
0925 05:17:40.791443 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.791445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.791456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.791513 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:40.792470 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.792981 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.794002 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.794204 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.794267 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.796609 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:40.796612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:40.796612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:40.796635 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:40.798319 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.798730 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.798731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.798732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.798797 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.799124 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.799125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.799125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.799280 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:40.799359 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.799767 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.799841 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 05:17:40.800223 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.790506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:39247)
0925 05:17:40.790682 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.790722 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.791337 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.791571 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.791574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.792167 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:40.792374 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.792379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.793067 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.793068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.793893 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.793895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.794294 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.794295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.794409 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.795043 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:40.795052 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:40.796500 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:40.798412 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.798419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.798422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.798656 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.798871 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.798872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.798872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.799056 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.799360 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:40.799367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:40.799577 (+ 210us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.799717 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.799916 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260925 05:17:40.800824 28002 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:40.800937 28002 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:40.801154 28002 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:40.801528 27677 ts_manager.cc:194] Registered new tserver with Master: 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
I20260925 05:17:40.801935 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:39247
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:17:40.823364 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:40.907673 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:40.907902 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:40.907927 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:40.909097 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:40.909134 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:40.909161 28007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:40.910593 28007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:40.910815 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:40.911087 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:40.916146 28015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.916162 28013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:40.916146 28012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:40.916623 28007 server_base.cc:1034] running on GCE node
I20260925 05:17:40.916880 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:40.917169 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:40.918347 28007 hybrid_clock.cc:648] HybridClock initialized: now 1790313460918333 us; error 30 us; skew 500 ppm
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:40.920970 28007 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:40.921381 28007 webserver.cc:460] Webserver started at http://127.13.160.131:41055/ using document root <none> and password file <none>
I20260925 05:17:40.921566 28007 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:40.921607 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:40.921697 28007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:40.922727 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "f76e845717d4420fbbb296f678fa2c3b"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "d5e3494b0114c8ff187ba44f44c79743"
server_key_iv: "21336ba476b723be2c1a4fb7b5f0f4f6"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.923125 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "f76e845717d4420fbbb296f678fa2c3b"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "d5e3494b0114c8ff187ba44f44c79743"
server_key_iv: "21336ba476b723be2c1a4fb7b5f0f4f6"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.924412 28007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:40.925112 28022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.925287 28007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:17:40.925359 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "f76e845717d4420fbbb296f678fa2c3b"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "d5e3494b0114c8ff187ba44f44c79743"
server_key_iv: "21336ba476b723be2c1a4fb7b5f0f4f6"
server_key_version: "encryptionkey@0"
I20260925 05:17:40.925412 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:40.934316 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:40.936399 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:40.936518 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:40.936715 28007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:40.937052 28007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:40.937085 28007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.937107 28007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:40.937156 28007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:40.943403 28007 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:40131
I20260925 05:17:40.943480 28135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:40131 every 8 connection(s)
I20260925 05:17:40.943825 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:17:40.948518 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28007
I20260925 05:17:40.948674 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:17:40.948866 13954 external_mini_cluster.cc:1428] Setting key ffc963612b3ee2d532518e656eedbd69
Sep 25 05:17:40 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313460, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:40.953536 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.944670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:60275 (local address 127.13.160.190:33621)
0925 05:17:40.944719 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.944724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.944860 (+ 136us) server_negotiation.cc:407] Connection header received
0925 05:17:40.945353 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.945354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.945364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.945428 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:40.946407 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.946946 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.948027 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.948213 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.948270 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.950523 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:40.950524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:40.950524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:40.950532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:40.952070 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.952508 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.952509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.952510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.952572 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.952827 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.952829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.952829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.952979 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:40.953059 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.953351 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.953439 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:17:40.953747 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.944564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:60275)
0925 05:17:40.944726 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.944770 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.945268 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.945513 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.945517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.946116 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:40.946321 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.946326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.947037 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.947038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.947938 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.947941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.948289 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.948290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.948390 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.949010 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:40.949019 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:40.950441 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:40.952175 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.952184 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.952186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.952428 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.952643 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:40.952644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:40.952645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:40.952757 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:40.953049 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:40.953057 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:40.953205 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.953315 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.953496 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260925 05:17:40.954212 28136 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:40.954331 28136 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:40.954528 28136 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:40.954981 27677 ts_manager.cc:194] Registered new tserver with Master: f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131)
I20260925 05:17:40.955365 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:60275
I20260925 05:17:40.960539 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:17:40.968084 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.961917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.13.160.190:33621)
0925 05:17:40.962005 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.962008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.962089 (+ 81us) server_negotiation.cc:407] Connection header received
0925 05:17:40.962137 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.962138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.962147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.962200 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:40.962729 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.963286 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.963990 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.964189 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.964239 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.965438 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:40.965439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:40.965440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:40.965446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:40.966803 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.967179 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.967180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.967180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.967232 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:40.967491 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:40.967492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:40.967493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:40.967649 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:40.967757 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.967881 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.967969 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 05:17:40.969642 27677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:17:40.971248 27677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:17:40.977932 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.973764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40971 (local address 127.0.0.1:49018)
0925 05:17:40.974138 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.974148 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.974191 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.974656 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.974657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.974659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:40.974704 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.974706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975660 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.975661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.976752 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.976754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.977677 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.977678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.977733 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.977745 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.977802 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.977841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":35,"thread_start_us":101,"threads_started":1}
I20260925 05:17:40.977982 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.974052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.13.160.129:40971)
0925 05:17:40.974501 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.974505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.974526 (+ 21us) server_negotiation.cc:407] Connection header received
0925 05:17:40.974566 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.974568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.974583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.974657 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:40.974752 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975549 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.976848 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.977593 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.977675 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.977731 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.977840 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.977889 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":77,"threads_started":1}
I20260925 05:17:40.978225 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.973558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.0.0.1:46638)
0925 05:17:40.973672 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.973714 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.973775 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.974333 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.974334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.974335 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:40.974368 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.974369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975683 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.975684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.977017 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.977019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.977973 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.977974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.978033 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.978038 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.978097 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.978146 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":89,"thread_start_us":65,"threads_started":1}
I20260925 05:17:40.978518 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.973952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.13.160.130:33817)
0925 05:17:40.974122 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.974127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.974139 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:17:40.974184 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.974187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.974202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.974328 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:40.974446 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975557 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.977094 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.977857 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.977926 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.977973 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.978364 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.978434 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260925 05:17:40.979210 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.973587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40131 (local address 127.0.0.1:44230)
0925 05:17:40.973726 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:40.973736 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:40.973790 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:40.974203 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:40.974207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:40.974209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:40.974278 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.974280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975682 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.975684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.977003 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:40.977006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.978944 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.978945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:40.979022 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.979025 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:17:40.979095 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 05:17:40.979139 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":91,"thread_start_us":93,"threads_started":1}
I20260925 05:17:40.979135 27802 tablet_service.cc:1467] Processing CreateTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d (DEFAULT_TABLE table=test-table [id=3200781d77cd4355a3b7bed4fea8900d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:40.979573 27802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b3c5aeda8d4814b86b1a0583c2389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:40.979528 27936 tablet_service.cc:1467] Processing CreateTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d (DEFAULT_TABLE table=test-table [id=3200781d77cd4355a3b7bed4fea8900d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:40.979854 27936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b3c5aeda8d4814b86b1a0583c2389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:40.982420 28153 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Bootstrap starting.
I20260925 05:17:40.982784 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:40.973749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.13.160.131:40131)
0925 05:17:40.973949 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:40.973954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:40.973966 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:17:40.974010 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:40.974013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:40.974030 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:40.974108 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:40.974362 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.975565 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.978023 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:40.978761 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:40.978828 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:40.978886 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:40.982615 (+ 3729us) server_negotiation.cc:299] Negotiation successful
0925 05:17:40.982680 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
I20260925 05:17:40.983191 28153 tablet_bootstrap.cc:654] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:40.983506 28070 tablet_service.cc:1467] Processing CreateTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d (DEFAULT_TABLE table=test-table [id=3200781d77cd4355a3b7bed4fea8900d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:40.983775 28153 log.cc:824] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:40.983819 28070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b3c5aeda8d4814b86b1a0583c2389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:40.984162 28154 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Bootstrap starting.
I20260925 05:17:40.984540 28153 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: No bootstrap required, opened a new log
I20260925 05:17:40.984596 28153 ts_tablet_manager.cc:1397] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:40.986200 28153 raft_consensus.cc:348] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.986342 28153 raft_consensus.cc:374] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:40.986366 28153 raft_consensus.cc:729] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ea74039083468798a8177f69115cc8, State: Initialized, Role: FOLLOWER
I20260925 05:17:40.986490 28153 consensus_queue.cc:260] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.986783 28153 ts_tablet_manager.cc:1428] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:40.986761 28156 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Bootstrap starting.
I20260925 05:17:40.986994 27868 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
I20260925 05:17:40.987448 28156 tablet_bootstrap.cc:654] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:40.987569 28154 tablet_bootstrap.cc:654] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:40.987987 28156 log.cc:824] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:40.988142 28154 log.cc:824] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:40.988834 28156 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: No bootstrap required, opened a new log
I20260925 05:17:40.988911 28156 ts_tablet_manager.cc:1397] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:17:40.988951 28154 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: No bootstrap required, opened a new log
I20260925 05:17:40.989054 28154 ts_tablet_manager.cc:1397] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 05:17:40.990365 28156 raft_consensus.cc:348] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.990535 28156 raft_consensus.cc:374] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:40.990581 28156 raft_consensus.cc:729] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76e845717d4420fbbb296f678fa2c3b, State: Initialized, Role: FOLLOWER
I20260925 05:17:40.990603 28154 raft_consensus.cc:348] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.990705 28156 consensus_queue.cc:260] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.990769 28154 raft_consensus.cc:374] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:40.990794 28154 raft_consensus.cc:729] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c5e28f6e2b4b7fb720dfab80114732, State: Initialized, Role: FOLLOWER
I20260925 05:17:40.990958 28154 consensus_queue.cc:260] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:40.991042 28156 ts_tablet_manager.cc:1428] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:17:40.991096 28136 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
I20260925 05:17:40.991279 28154 ts_tablet_manager.cc:1428] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 05:17:40.991425 28002 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
W20260925 05:17:41.040545 28003 tablet.cc:2369] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:17:41.084762 28160 raft_consensus.cc:482] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:41.084901 28160 raft_consensus.cc:504] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.085353 28160 leader_election.cc:288] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817), 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971)
I20260925 05:17:41.089413 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.085862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:34995 (local address 127.13.160.129:40971)
0925 05:17:41.085947 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.085949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.085975 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:17:41.086033 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.086034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.086041 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.086109 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:41.086239 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.086961 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.088490 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.089040 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.089099 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.089106 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.089283 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.089333 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 05:17:41.089430 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.085741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40971 (local address 127.13.160.131:34995)
0925 05:17:41.085913 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.085923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.085967 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.086112 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.086113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.086115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:41.086175 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.086176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.087049 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.087050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.088387 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.088391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.089122 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.089123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.089204 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.089210 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.089266 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.089302 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":87,"thread_start_us":113,"threads_started":1}
I20260925 05:17:41.089519 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.085714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:33637 (local address 127.13.160.130:33817)
0925 05:17:41.085773 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.085775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.085809 (+ 34us) server_negotiation.cc:407] Connection header received
0925 05:17:41.085856 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.085858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.085868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.085917 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:41.086116 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.086929 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.088507 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.089103 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.089179 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.089190 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.089391 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.089446 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 05:17:41.089751 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.085590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.13.160.131:33637)
0925 05:17:41.085656 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.085669 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.085765 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.085944 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.085944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.085946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:41.086025 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.086027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.087060 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.087060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.088393 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.088395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.089196 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.089197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.089296 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.089300 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.089357 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.089406 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":9}
I20260925 05:17:41.089879 27822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b3c5aeda8d4814b86b1a0583c2389d" candidate_uuid: "f76e845717d4420fbbb296f678fa2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ea74039083468798a8177f69115cc8" is_pre_election: true
I20260925 05:17:41.090049 27822 raft_consensus.cc:2454] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76e845717d4420fbbb296f678fa2c3b in term 0.
I20260925 05:17:41.090461 28023 leader_election.cc:302] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82ea74039083468798a8177f69115cc8, f76e845717d4420fbbb296f678fa2c3b; no voters:
I20260925 05:17:41.090543 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b3c5aeda8d4814b86b1a0583c2389d" candidate_uuid: "f76e845717d4420fbbb296f678fa2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c5e28f6e2b4b7fb720dfab80114732" is_pre_election: true
I20260925 05:17:41.090605 28160 raft_consensus.cc:2781] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:17:41.090662 28160 raft_consensus.cc:482] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:17:41.090698 28160 raft_consensus.cc:3037] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.090708 27956 raft_consensus.cc:2454] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76e845717d4420fbbb296f678fa2c3b in term 0.
I20260925 05:17:41.091509 28160 raft_consensus.cc:504] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.091666 28160 leader_election.cc:288] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [CANDIDATE]: Term 1 election: Requested vote from peers 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817), 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971)
I20260925 05:17:41.091872 27822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b3c5aeda8d4814b86b1a0583c2389d" candidate_uuid: "f76e845717d4420fbbb296f678fa2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ea74039083468798a8177f69115cc8"
I20260925 05:17:41.091872 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b3c5aeda8d4814b86b1a0583c2389d" candidate_uuid: "f76e845717d4420fbbb296f678fa2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c5e28f6e2b4b7fb720dfab80114732"
I20260925 05:17:41.091965 27956 raft_consensus.cc:3037] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.091965 27822 raft_consensus.cc:3037] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.092926 27822 raft_consensus.cc:2454] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76e845717d4420fbbb296f678fa2c3b in term 1.
I20260925 05:17:41.092934 27956 raft_consensus.cc:2454] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76e845717d4420fbbb296f678fa2c3b in term 1.
I20260925 05:17:41.093103 28023 leader_election.cc:302] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82ea74039083468798a8177f69115cc8, f76e845717d4420fbbb296f678fa2c3b; no voters:
I20260925 05:17:41.093201 28160 raft_consensus.cc:2781] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:41.093366 28160 raft_consensus.cc:686] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 1 LEADER]: Becoming Leader. State: Replica: f76e845717d4420fbbb296f678fa2c3b, State: Running, Role: LEADER
I20260925 05:17:41.093482 28160 consensus_queue.cc:237] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.094401 27677 catalog_manager.cc:5495] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to f76e845717d4420fbbb296f678fa2c3b (127.13.160.131). New cstate: current_term: 1 leader_uuid: "f76e845717d4420fbbb296f678fa2c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } health_report { overall_health: HEALTHY } } }
W20260925 05:17:41.109009 27869 tablet.cc:2369] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:17:41.122331 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.119941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.13.160.131:40131)
0925 05:17:41.120020 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.120022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.120041 (+ 19us) server_negotiation.cc:407] Connection header received
0925 05:17:41.120082 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.120083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.120103 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.120176 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:17:41.120318 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.120811 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.121556 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.121744 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.121800 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.121945 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:17:41.122058 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:17:41.122128 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.122160 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.122179 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 05:17:41.124409 27822 raft_consensus.cc:1261] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Refusing update from remote peer f76e845717d4420fbbb296f678fa2c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:41.124409 27956 raft_consensus.cc:1261] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Refusing update from remote peer f76e845717d4420fbbb296f678fa2c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:41.124799 28163 consensus_queue.cc:1035] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:41.124943 28160 consensus_queue.cc:1035] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:41.127738 28167 mvcc.cc:205] Tried to move back new op lower bound from 7333123936762433536 to 7333123936639750144. Current Snapshot: MvccSnapshot[applied={T|T < 7333123936762433536}]
I20260925 05:17:41.130124 27677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:41.130240 27677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:41.131080 27677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Sending DeleteTablet for 3 replicas of tablet 84b3c5aeda8d4814b86b1a0583c2389d
I20260925 05:17:41.131417 28070 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:44230
I20260925 05:17:41.131415 27936 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:46638
I20260925 05:17:41.131486 27802 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:49018
I20260925 05:17:41.131688 28178 tablet_replica.cc:330] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: stopping tablet replica
I20260925 05:17:41.131740 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27739
I20260925 05:17:41.131842 28178 raft_consensus.cc:2229] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.131942 28178 raft_consensus.cc:2258] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.131816 28176 tablet_replica.cc:330] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: stopping tablet replica
I20260925 05:17:41.132453 28176 raft_consensus.cc:2229] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:17:41.132545 28176 raft_consensus.cc:2258] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.132944 28176 ts_tablet_manager.cc:1905] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:41.135058 28176 ts_tablet_manager.cc:1918] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:41.135139 28176 log.cc:1196] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/84b3c5aeda8d4814b86b1a0583c2389d
I20260925 05:17:41.135455 28176 ts_tablet_manager.cc:1939] T 84b3c5aeda8d4814b86b1a0583c2389d P f76e845717d4420fbbb296f678fa2c3b: Deleting consensus metadata
I20260925 05:17:41.136025 27663 catalog_manager.cc:4841] TS f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): tablet 84b3c5aeda8d4814b86b1a0583c2389d (table test-table [id=3200781d77cd4355a3b7bed4fea8900d]) successfully deleted
W20260925 05:17:41.137487 27661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:40971 (error 108)
I20260925 05:17:41.138089 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.137943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40971 (local address 127.0.0.1:49020)
0925 05:17:41.138004 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.138049 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 05:17:41.138126 28177 tablet_replica.cc:330] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: stopping tablet replica
I20260925 05:17:41.138116 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27873
I20260925 05:17:41.138245 28177 raft_consensus.cc:2229] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.138398 28177 raft_consensus.cc:2258] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 05:17:41.143123 27661 catalog_manager.cc:4584] TS 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84b3c5aeda8d4814b86b1a0583c2389d: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40971: connect: Connection refused (error 111)
I20260925 05:17:41.145236 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28007
I20260925 05:17:41.145273 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.143632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.0.0.1:46654)
0925 05:17:41.145182 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.145226 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:33817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1492}
I20260925 05:17:41.150152 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 27645
I20260925 05:17:41.155894 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--webserver_interface=127.13.160.190
--webserver_port=43753
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:41.239069 28180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:41.239336 28180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:41.239378 28180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:41.240573 28180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:41.240648 28180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:41.240674 28180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:41.240702 28180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:41.240736 28180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:41.242254 28180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:41.242564 28180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:41.242810 28180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:41.251700 28186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.251703 28187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.251700 28189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.251957 28180 server_base.cc:1034] running on GCE node
I20260925 05:17:41.252226 28180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:41.252579 28180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:41.253755 28180 hybrid_clock.cc:648] HybridClock initialized: now 1790313461253734 us; error 39 us; skew 500 ppm
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:41.261631 28180 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:41.262192 28180 webserver.cc:460] Webserver started at http://127.13.160.190:43753/ using document root <none> and password file <none>
I20260925 05:17:41.262454 28180 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:41.262557 28180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:41.264082 28180 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.264786 28196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.264959 28180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:41.265077 28180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "6083a35baf8e4919b1d17843ceda322e"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "eca67cfdd9d0b326b84a9120aea6613c"
server_key_iv: "458f54edd6c38bb415690b45c1d8caf5"
server_key_version: "encryptionkey@0"
I20260925 05:17:41.265520 28180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:41.279377 28180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:41.281410 28180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:41.281603 28180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:41.285998 28180 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33621
I20260925 05:17:41.286036 28248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33621 every 8 connection(s)
I20260925 05:17:41.286475 28180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:41.288235 28249 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:17:41.288985 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap starting.
I20260925 05:17:41.291345 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28180
I20260925 05:17:41.291661 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:40971
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=44405
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:41.297689 28249 log.cc:824] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:41.299896 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:41.300323 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap complete.
I20260925 05:17:41.302722 28249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.303058 28249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.303259 28249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.303339 28249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:41.303398 28249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:41.303447 28249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:17:41.304563 28249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.304702 28249 leader_election.cc:302] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6083a35baf8e4919b1d17843ceda322e; no voters:
I20260925 05:17:41.304919 28249 leader_election.cc:288] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 05:17:41.304960 28253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Leader election won for term 2
I20260925 05:17:41.305095 28253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 LEADER]: Becoming Leader. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Running, Role: LEADER
I20260925 05:17:41.305255 28253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.305253 28249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:41.305658 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:41.305739 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:41.305873 28254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6083a35baf8e4919b1d17843ceda322e. Latest consensus state: current_term: 2 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:41.305970 28254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:41.306134 28257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:41.307103 28257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:41.307278 28257 catalog_manager.cc:1259] Loaded cluster ID: f57ca2ccd70d4024ad87cac2d3365b08
I20260925 05:17:41.307546 28257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:17:41.307816 28257 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 05:17:41.311210 28257 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 05:17:41.313324 28257 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:41.313565 28257 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Loaded TSK: 0
I20260925 05:17:41.313797 28257 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:17:41.397379 28251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:41.397603 28251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:41.397629 28251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:41.398869 28251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:41.398985 28251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:41.399020 28251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:41.400571 28251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:40971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=44405
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:41.400821 28251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:41.401089 28251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:41.407253 28276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.407253 28277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.407408 28279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.407745 28251 server_base.cc:1034] running on GCE node
I20260925 05:17:41.407965 28251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:41.408228 28251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:41.409394 28251 hybrid_clock.cc:648] HybridClock initialized: now 1790313461409383 us; error 34 us; skew 500 ppm
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:41.412349 28251 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:41.412978 28251 webserver.cc:460] Webserver started at http://127.13.160.129:44405/ using document root <none> and password file <none>
I20260925 05:17:41.413264 28251 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:41.413362 28251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:41.415364 28251 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:17:41.416289 28286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.416517 28251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.416606 28251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "82ea74039083468798a8177f69115cc8"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "5bb24669033a900966b27a6b099d5857"
server_key_iv: "e097f47139108c8b6780468a025a76de"
server_key_version: "encryptionkey@0"
I20260925 05:17:41.417034 28251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:41.434798 28251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:41.437357 28251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:41.437479 28251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:41.437777 28251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:41.438256 28293 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 05:17:41.440304 28251 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 05:17:41.440377 28251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 05:17:41.440426 28251 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 05:17:41.441092 28251 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 05:17:41.441150 28251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:41.441239 28293 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Bootstrap starting.
I20260925 05:17:41.448698 28251 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40971
I20260925 05:17:41.449111 28251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:17:41.451709 28400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40971 every 8 connection(s)
I20260925 05:17:41.452993 28293 log.cc:824] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:41.455473 28293 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:41.455806 28293 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Bootstrap complete.
I20260925 05:17:41.457077 28293 ts_tablet_manager.cc:1397] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260925 05:17:41.457444 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28251
I20260925 05:17:41.457756 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:33817
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=39081
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:41.458343 28293 raft_consensus.cc:348] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.458647 28293 raft_consensus.cc:729] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ea74039083468798a8177f69115cc8, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.458763 28293 consensus_queue.cc:260] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.459084 28293 ts_tablet_manager.cc:1428] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:41.464859 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.449829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:37447)
0925 05:17:41.450026 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.450072 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.450536 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.451496 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.451500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.452021 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:41.452217 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.452223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.453550 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.453551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.454783 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.454787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.455647 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.455648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.455815 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.456486 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:41.456496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:41.460592 (+ 4096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:41.462789 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.462797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.462799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.463170 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.463489 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.463490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.463490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.463626 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.464129 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:41.464135 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:41.464310 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.464430 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.464635 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260925 05:17:41.464943 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.449963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:37447 (local address 127.13.160.190:33621)
0925 05:17:41.451121 (+ 1158us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.451129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.451156 (+ 27us) server_negotiation.cc:407] Connection header received
0925 05:17:41.451264 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.451269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.451310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.451448 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.452339 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.453400 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.454920 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.455515 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.456105 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.460816 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.460820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.460821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.460833 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.462698 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.463269 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.463273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.463273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.463406 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.463702 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.463703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.463704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.464045 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.464138 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.464510 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.464693 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1111,"thread_start_us":100,"threads_started":1}
I20260925 05:17:41.465737 28401 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:41.465844 28401 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:41.466046 28401 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:41.466961 28213 ts_manager.cc:194] Registered new tserver with Master: 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971)
I20260925 05:17:41.468405 28213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:37447
I20260925 05:17:41.470162 28401 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.129@KRBTEST.COM
I20260925 05:17:41.480732 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.468065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.13.160.129:40971)
0925 05:17:41.468444 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.468448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.468894 (+ 446us) server_negotiation.cc:407] Connection header received
0925 05:17:41.468933 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.468935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.468952 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.469014 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.470034 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.471539 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.472980 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.473464 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.473551 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.477180 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.477190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.477191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.477199 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.479260 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.479780 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.479784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.479784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.479853 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.480074 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.480074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.480075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.480252 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.480331 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.480531 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.480623 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":72,"threads_started":1}
I20260925 05:17:41.480764 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.467763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40971 (local address 127.0.0.1:49030)
0925 05:17:41.468477 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.468516 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.468596 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.469084 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.469090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.469448 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:41.469550 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.469553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.471640 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.471642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.472865 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.472869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.473556 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.473557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.473649 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.475291 (+ 1642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:41.475296 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:41.477080 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:41.479361 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.479370 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.479373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.479681 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.479932 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.479932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.479933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.480005 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.480321 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:41.480328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:41.480389 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.480486 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.480625 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":133,"threads_started":1}
I20260925 05:17:41.481554 28325 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:49030
I20260925 05:17:41.482206 28410 tablet_replica.cc:330] stopping tablet replica
I20260925 05:17:41.482326 28410 raft_consensus.cc:2229] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.482401 28410 raft_consensus.cc:2258] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.482780 28410 ts_tablet_manager.cc:1905] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:41.485572 28410 ts_tablet_manager.cc:1918] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:41.485661 28410 log.cc:1196] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/84b3c5aeda8d4814b86b1a0583c2389d
I20260925 05:17:41.485967 28410 ts_tablet_manager.cc:1939] T 84b3c5aeda8d4814b86b1a0583c2389d P 82ea74039083468798a8177f69115cc8: Deleting consensus metadata
I20260925 05:17:41.486547 28197 catalog_manager.cc:4841] TS 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971): tablet 84b3c5aeda8d4814b86b1a0583c2389d (table test-table [id=3200781d77cd4355a3b7bed4fea8900d]) successfully deleted
W20260925 05:17:41.550127 28407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:41.550403 28407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:41.550446 28407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:41.551769 28407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:41.551848 28407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:41.551895 28407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:41.553443 28407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:33817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=39081
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:41.553737 28407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:41.553992 28407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:41.559238 28415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.559279 28418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.559402 28407 server_base.cc:1034] running on GCE node
W20260925 05:17:41.559615 28416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.559831 28407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:41.560052 28407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:41.561210 28407 hybrid_clock.cc:648] HybridClock initialized: now 1790313461561189 us; error 27 us; skew 500 ppm
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:41.563804 28407 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:41.564325 28407 webserver.cc:460] Webserver started at http://127.13.160.130:39081/ using document root <none> and password file <none>
I20260925 05:17:41.564565 28407 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:41.564661 28407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:41.566062 28407 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:41.566740 28425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.566959 28407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:41.567058 28407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "19c5e28f6e2b4b7fb720dfab80114732"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "83edcb6b10e2235d1bec0321dfc5aeda"
server_key_iv: "6b813c2c857c97a23ce26d013261e667"
server_key_version: "encryptionkey@0"
I20260925 05:17:41.567497 28407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:41.582662 28407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:41.584793 28407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:41.584980 28407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:41.585244 28407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:41.585736 28432 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 05:17:41.587442 28407 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 05:17:41.587512 28407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 05:17:41.587558 28407 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 05:17:41.588225 28407 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 05:17:41.588282 28407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:41.588340 28432 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Bootstrap starting.
I20260925 05:17:41.595726 28407 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:33817
I20260925 05:17:41.595777 28539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:33817 every 8 connection(s)
I20260925 05:17:41.596141 28407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:17:41.596477 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28407
I20260925 05:17:41.596781 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:40131
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=41055
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:41.597975 28432 log.cc:824] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:41.600967 28432 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:41.601274 28432 tablet_bootstrap.cc:492] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Bootstrap complete.
I20260925 05:17:41.602229 28432 ts_tablet_manager.cc:1397] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.004s
I20260925 05:17:41.603749 28432 raft_consensus.cc:348] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.604048 28432 raft_consensus.cc:729] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c5e28f6e2b4b7fb720dfab80114732, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.604194 28432 consensus_queue.cc:260] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } } peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } }
I20260925 05:17:41.604493 28432 ts_tablet_manager.cc:1428] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:41.609324 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.598407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:38259 (local address 127.13.160.190:33621)
0925 05:17:41.598483 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.598486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.598741 (+ 255us) server_negotiation.cc:407] Connection header received
0925 05:17:41.599574 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.599576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.599594 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.599670 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.600577 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.601336 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.603048 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.603360 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.603443 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.606206 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.606207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.606208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.606217 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.607816 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.608267 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.608269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.608269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.608332 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.608618 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.608619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.608619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.608751 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.608827 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.609124 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.609226 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 05:17:41.609526 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.598363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:38259)
0925 05:17:41.598587 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.598642 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.599436 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.599752 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.599756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.600171 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:41.600460 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.600467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.601424 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.601426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.602926 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.602928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.603478 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.603480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.603646 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.604300 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:41.604312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:41.606091 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:41.607923 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.607932 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.607934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.608184 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.608409 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.608410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.608410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.608541 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.608818 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:41.608825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:41.608985 (+ 160us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.609105 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.609282 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260925 05:17:41.610136 28540 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:41.610247 28540 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:41.610437 28540 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:41.611070 28213 ts_manager.cc:194] Registered new tserver with Master: 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
I20260925 05:17:41.611675 28213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:38259
I20260925 05:17:41.612540 28540 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.130@KRBTEST.COM
I20260925 05:17:41.619755 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.611378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.0.0.1:46664)
0925 05:17:41.611430 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.611440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.611483 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.611893 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.611894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.612061 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:41.612105 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.612107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.613492 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.613493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.614069 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.614071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.614618 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.614619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.614692 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.615222 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:41.615228 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:41.616743 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:41.618534 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.618536 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.618536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.618723 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.618995 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.618996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.618996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.619059 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.619374 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:41.619375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:41.619425 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.619522 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.619650 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 05:17:41.619776 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.611544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.13.160.130:33817)
0925 05:17:41.611719 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.611723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.611734 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:17:41.611773 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.611775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.611790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.612170 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.612416 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.613407 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.614168 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.614495 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.614616 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.616834 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.616844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.616845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.616852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.618422 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.618808 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.618812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.618813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.618882 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.619133 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.619134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.619134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.619292 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.619385 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.619561 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.619652 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260925 05:17:41.620483 28474 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b3c5aeda8d4814b86b1a0583c2389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:46664
I20260925 05:17:41.621073 28547 tablet_replica.cc:330] stopping tablet replica
I20260925 05:17:41.621177 28547 raft_consensus.cc:2229] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.621244 28547 raft_consensus.cc:2258] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.621565 28547 ts_tablet_manager.cc:1905] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:41.624089 28547 ts_tablet_manager.cc:1918] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:41.624173 28547 log.cc:1196] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/84b3c5aeda8d4814b86b1a0583c2389d
I20260925 05:17:41.624392 28547 ts_tablet_manager.cc:1939] T 84b3c5aeda8d4814b86b1a0583c2389d P 19c5e28f6e2b4b7fb720dfab80114732: Deleting consensus metadata
I20260925 05:17:41.624902 28197 catalog_manager.cc:4841] TS 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): tablet 84b3c5aeda8d4814b86b1a0583c2389d (table test-table [id=3200781d77cd4355a3b7bed4fea8900d]) successfully deleted
W20260925 05:17:41.685921 28542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:41.686192 28542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:41.686232 28542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:41.687415 28542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:41.687493 28542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:41.687538 28542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:41.689037 28542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:40131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=41055
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:41.689329 28542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:41.689577 28542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:41.694680 28553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.694742 28555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.694748 28552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.694828 28542 server_base.cc:1034] running on GCE node
I20260925 05:17:41.695166 28542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:41.695380 28542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:41.696564 28542 hybrid_clock.cc:648] HybridClock initialized: now 1790313461696546 us; error 49 us; skew 500 ppm
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:41.699288 28542 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:41.699793 28542 webserver.cc:460] Webserver started at http://127.13.160.131:41055/ using document root <none> and password file <none>
I20260925 05:17:41.700049 28542 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:41.700130 28542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:41.701506 28542 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 05:17:41.702171 28562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.702386 28542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.702476 28542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "f76e845717d4420fbbb296f678fa2c3b"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "d5e3494b0114c8ff187ba44f44c79743"
server_key_iv: "21336ba476b723be2c1a4fb7b5f0f4f6"
server_key_version: "encryptionkey@0"
I20260925 05:17:41.702811 28542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:41.716732 28542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:41.718665 28542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:41.718787 28542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:41.719069 28542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:41.719449 28542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:41.719486 28542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.719511 28542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:41.719522 28542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.725961 28542 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:40131
I20260925 05:17:41.726022 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:40131 every 8 connection(s)
I20260925 05:17:41.726465 28542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:17:41.732771 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28542
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:41.738296 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.727698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:42341 (local address 127.13.160.190:33621)
0925 05:17:41.727797 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.727799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.728902 (+ 1103us) server_negotiation.cc:407] Connection header received
0925 05:17:41.729409 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.729411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.729422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.729483 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.730472 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.731038 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.732034 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.732223 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.732279 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.734968 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.734969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.734970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.734976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.736603 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.737147 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.737149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.737149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.737209 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.737538 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.737539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.737540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.737685 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.737759 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.738113 (+ 354us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.738196 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:17:41.738502 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.727370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:42341)
0925 05:17:41.728749 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.728803 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.729317 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.729543 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.729547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.730167 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:41.730376 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.730381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.731124 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.731124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.731941 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.731943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.732301 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.732302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.732410 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.733013 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:41.733026 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:41.734834 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:41.736684 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.736693 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.736697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.737050 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.737286 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:41.737287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:41.737288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:41.737440 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:41.737759 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:41.737766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:41.737956 (+ 190us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.738064 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.738227 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":85,"threads_started":1}
I20260925 05:17:41.739290 28676 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33621
I20260925 05:17:41.739413 28676 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:41.739642 28676 heartbeater.cc:511] Master 127.13.160.190:33621 requested a full tablet report, sending...
I20260925 05:17:41.740103 28213 ts_manager.cc:194] Registered new tserver with Master: f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131)
I20260925 05:17:41.740383 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.733243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.13.160.190:33621)
0925 05:17:41.733361 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.733363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.733375 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:17:41.733461 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.733462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.733474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.733544 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:41.734200 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.734743 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.735734 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.735943 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.736036 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.737410 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:41.737411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:41.737411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:41.737416 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:41.739178 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.739590 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.739590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.739591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.739642 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:41.739857 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:41.739858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:41.739858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:41.740005 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:41.740117 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.740227 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.740320 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260925 05:17:41.740605 28213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:42341
I20260925 05:17:41.741441 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:17:41.744010 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.741760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.13.160.190:33621)
0925 05:17:41.741814 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.741815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.741925 (+ 110us) server_negotiation.cc:407] Connection header received
0925 05:17:41.741972 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.741973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.741999 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.742054 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:17:41.742129 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.742578 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.743350 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.743541 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.743594 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.743718 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:17:41.743808 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:17:41.743869 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.743932 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.743950 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 05:17:41.744292 28213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:17:41.745966 28213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:17:41.748180 28474 tablet_service.cc:1467] Processing CreateTablet for tablet c753c56b0df74badadaec4f91dca2548 (DEFAULT_TABLE table=test-table [id=0d7134a9fa204e0a89801b08373d495e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:41.748392 28474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c753c56b0df74badadaec4f91dca2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:41.749815 28432 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Bootstrap starting.
I20260925 05:17:41.749944 28325 tablet_service.cc:1467] Processing CreateTablet for tablet c753c56b0df74badadaec4f91dca2548 (DEFAULT_TABLE table=test-table [id=0d7134a9fa204e0a89801b08373d495e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:41.750124 28325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c753c56b0df74badadaec4f91dca2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:41.750196 28432 tablet_bootstrap.cc:654] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:41.750731 28432 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: No bootstrap required, opened a new log
I20260925 05:17:41.750787 28432 ts_tablet_manager.cc:1397] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:41.751179 28432 raft_consensus.cc:348] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.751266 28432 raft_consensus.cc:374] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:41.751310 28432 raft_consensus.cc:729] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c5e28f6e2b4b7fb720dfab80114732, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.751361 28432 consensus_queue.cc:260] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.751441 28432 ts_tablet_manager.cc:1428] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.751688 28293 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Bootstrap starting.
I20260925 05:17:41.751880 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.747931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40131 (local address 127.0.0.1:44250)
0925 05:17:41.747966 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.747979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.748016 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.748502 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.748503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.748505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:41.748560 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.748562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.749461 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.749462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.750592 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.750596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.751595 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.751596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.751664 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.751689 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.751755 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.751803 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 05:17:41.752023 28293 tablet_bootstrap.cc:654] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:41.752105 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.748095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.13.160.131:40131)
0925 05:17:41.748305 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.748310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.748321 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:17:41.748361 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.748363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.748386 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.748620 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:41.748665 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.749355 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.750676 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.751503 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.751584 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.751647 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.751968 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.752018 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260925 05:17:41.752576 28293 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: No bootstrap required, opened a new log
I20260925 05:17:41.752642 28293 ts_tablet_manager.cc:1397] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:41.752835 28610 tablet_service.cc:1467] Processing CreateTablet for tablet c753c56b0df74badadaec4f91dca2548 (DEFAULT_TABLE table=test-table [id=0d7134a9fa204e0a89801b08373d495e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:41.752945 28293 raft_consensus.cc:348] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.753044 28293 raft_consensus.cc:374] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:41.753098 28293 raft_consensus.cc:729] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ea74039083468798a8177f69115cc8, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.753151 28293 consensus_queue.cc:260] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.753159 28610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c753c56b0df74badadaec4f91dca2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:41.753242 28293 ts_tablet_manager.cc:1428] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:41.756366 28685 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Bootstrap starting.
I20260925 05:17:41.756947 28685 tablet_bootstrap.cc:654] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:41.757416 28685 log.cc:824] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:41.758249 28685 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: No bootstrap required, opened a new log
I20260925 05:17:41.758330 28685 ts_tablet_manager.cc:1397] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:17:41.759864 28685 raft_consensus.cc:348] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.760030 28685 raft_consensus.cc:374] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:41.760075 28685 raft_consensus.cc:729] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76e845717d4420fbbb296f678fa2c3b, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.760210 28685 consensus_queue.cc:260] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.760455 28406 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:41.760528 28685 ts_tablet_manager.cc:1428] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 05:17:41.760535 28676 heartbeater.cc:503] Master 127.13.160.190:33621 was elected leader, sending a full tablet report...
I20260925 05:17:41.760581 28406 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.760941 28406 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
I20260925 05:17:41.764016 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.761143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:56489 (local address 127.13.160.131:40131)
0925 05:17:41.761182 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.761184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.761409 (+ 225us) server_negotiation.cc:407] Connection header received
0925 05:17:41.761507 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.761508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.761516 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.761571 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:41.761779 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.762261 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763250 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.763622 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763677 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.763684 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.763906 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.763955 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:17:41.764070 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.761173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:58813 (local address 127.13.160.130:33817)
0925 05:17:41.761219 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.761221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.761230 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:17:41.761262 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.761263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.761286 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.761359 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:41.761639 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.762170 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763255 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.763726 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763801 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.763883 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.763955 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.764002 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:17:41.764078 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.761077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.13.160.129:58813)
0925 05:17:41.761136 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.761147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.761196 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.761480 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.761481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.761482 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:41.761554 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.761556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.762252 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.762253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.763160 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.763163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.763804 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.763863 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.763885 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.763948 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.763994 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":26}
I20260925 05:17:41.764395 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.761080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40131 (local address 127.13.160.129:56489)
0925 05:17:41.761354 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.761364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:41.761434 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:41.761576 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:41.761577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:41.761578 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:41.761695 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.761698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.762336 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.762336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.763166 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:41.763168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.763721 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.763722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:41.763810 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.763834 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 05:17:41.763891 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:17:41.763939 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":119,"thread_start_us":96,"threads_started":1}
I20260925 05:17:41.764402 28494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c753c56b0df74badadaec4f91dca2548" candidate_uuid: "82ea74039083468798a8177f69115cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c5e28f6e2b4b7fb720dfab80114732" is_pre_election: true
I20260925 05:17:41.764569 28494 raft_consensus.cc:2454] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ea74039083468798a8177f69115cc8 in term 0.
I20260925 05:17:41.764741 28630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c753c56b0df74badadaec4f91dca2548" candidate_uuid: "82ea74039083468798a8177f69115cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76e845717d4420fbbb296f678fa2c3b" is_pre_election: true
I20260925 05:17:41.764820 28287 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19c5e28f6e2b4b7fb720dfab80114732, 82ea74039083468798a8177f69115cc8; no voters:
I20260925 05:17:41.764880 28630 raft_consensus.cc:2454] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ea74039083468798a8177f69115cc8 in term 0.
I20260925 05:17:41.764935 28406 raft_consensus.cc:2781] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:17:41.764999 28406 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:17:41.765028 28406 raft_consensus.cc:3037] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.765735 28406 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.765890 28406 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 1 election: Requested vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
I20260925 05:17:41.766036 28630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c753c56b0df74badadaec4f91dca2548" candidate_uuid: "82ea74039083468798a8177f69115cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76e845717d4420fbbb296f678fa2c3b"
I20260925 05:17:41.766109 28630 raft_consensus.cc:3037] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.766074 28494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c753c56b0df74badadaec4f91dca2548" candidate_uuid: "82ea74039083468798a8177f69115cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c5e28f6e2b4b7fb720dfab80114732"
I20260925 05:17:41.766182 28494 raft_consensus.cc:3037] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:41.766826 28630 raft_consensus.cc:2454] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ea74039083468798a8177f69115cc8 in term 1.
I20260925 05:17:41.767010 28494 raft_consensus.cc:2454] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ea74039083468798a8177f69115cc8 in term 1.
I20260925 05:17:41.767040 28289 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82ea74039083468798a8177f69115cc8, f76e845717d4420fbbb296f678fa2c3b; no voters:
I20260925 05:17:41.767117 28406 raft_consensus.cc:2781] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:41.767261 28406 raft_consensus.cc:686] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 LEADER]: Becoming Leader. State: Replica: 82ea74039083468798a8177f69115cc8, State: Running, Role: LEADER
I20260925 05:17:41.767381 28406 consensus_queue.cc:237] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:41.768213 28212 catalog_manager.cc:5495] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82ea74039083468798a8177f69115cc8 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "82ea74039083468798a8177f69115cc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } health_report { overall_health: HEALTHY } } }
I20260925 05:17:41.778875 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.776462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.13.160.129:40971)
0925 05:17:41.776529 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:41.776531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:41.776541 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:41.776588 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:41.776589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:41.776606 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:41.776685 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:17:41.776791 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.777287 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.778036 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:41.778251 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:41.778338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:41.778518 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:17:41.778627 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:17:41.778706 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:41.778757 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 05:17:41.778780 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:17:41.780470 28494 raft_consensus.cc:1261] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Refusing update from remote peer 82ea74039083468798a8177f69115cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:41.780483 28630 raft_consensus.cc:1261] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Refusing update from remote peer 82ea74039083468798a8177f69115cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:41.780864 28689 consensus_queue.cc:1035] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:41.781011 28406 consensus_queue.cc:1035] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:41.783232 28692 mvcc.cc:205] Tried to move back new op lower bound from 7333123939450163200 to 7333123939399872512. Current Snapshot: MvccSnapshot[applied={T|T < 7333123939450163200}]
I20260925 05:17:41.783775 28693 mvcc.cc:205] Tried to move back new op lower bound from 7333123939450163200 to 7333123939399872512. Current Snapshot: MvccSnapshot[applied={T|T < 7333123939450163200}]
I20260925 05:17:41.785789 28213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:41.785912 28213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:41.786721 28213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Sending DeleteTablet for 3 replicas of tablet c753c56b0df74badadaec4f91dca2548
I20260925 05:17:41.787101 28325 tablet_service.cc:1514] Processing DeleteTablet for tablet c753c56b0df74badadaec4f91dca2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:49030
I20260925 05:17:41.787109 28610 tablet_service.cc:1514] Processing DeleteTablet for tablet c753c56b0df74badadaec4f91dca2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:44250
I20260925 05:17:41.787209 28474 tablet_service.cc:1514] Processing DeleteTablet for tablet c753c56b0df74badadaec4f91dca2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:41 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:46664
I20260925 05:17:41.787303 28410 tablet_replica.cc:330] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: stopping tablet replica
I20260925 05:17:41.787365 28410 raft_consensus.cc:2229] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:17:41.787438 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28251
I20260925 05:17:41.787456 28410 raft_consensus.cc:2258] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.787542 28547 tablet_replica.cc:330] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: stopping tablet replica
I20260925 05:17:41.787602 28547 raft_consensus.cc:2229] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.787657 28547 raft_consensus.cc:2258] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.788213 28702 tablet_replica.cc:330] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: stopping tablet replica
I20260925 05:17:41.788322 28702 raft_consensus.cc:2229] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:41.788359 28547 ts_tablet_manager.cc:1905] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:41.788437 28702 raft_consensus.cc:2258] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:41.788869 28702 ts_tablet_manager.cc:1905] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:41.790014 28547 ts_tablet_manager.cc:1918] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:41.790091 28547 log.cc:1196] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/c753c56b0df74badadaec4f91dca2548
I20260925 05:17:41.790308 28547 ts_tablet_manager.cc:1939] T c753c56b0df74badadaec4f91dca2548 P 19c5e28f6e2b4b7fb720dfab80114732: Deleting consensus metadata
I20260925 05:17:41.790685 28197 catalog_manager.cc:4841] TS 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): tablet c753c56b0df74badadaec4f91dca2548 (table test-table [id=0d7134a9fa204e0a89801b08373d495e]) successfully deleted
I20260925 05:17:41.790794 28702 ts_tablet_manager.cc:1918] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:41.790874 28702 log.cc:1196] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/c753c56b0df74badadaec4f91dca2548
I20260925 05:17:41.791174 28702 ts_tablet_manager.cc:1939] T c753c56b0df74badadaec4f91dca2548 P f76e845717d4420fbbb296f678fa2c3b: Deleting consensus metadata
I20260925 05:17:41.791626 28199 catalog_manager.cc:4841] TS f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): tablet c753c56b0df74badadaec4f91dca2548 (table test-table [id=0d7134a9fa204e0a89801b08373d495e]) successfully deleted
W20260925 05:17:41.794463 28197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:40971 (error 108)
I20260925 05:17:41.794916 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28407
I20260925 05:17:41.795022 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:41.794845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40971 (local address 127.0.0.1:49036)
0925 05:17:41.794930 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:41.794982 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 05:17:41.799597 28197 catalog_manager.cc:4584] TS 82ea74039083468798a8177f69115cc8 (127.13.160.129:40971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c753c56b0df74badadaec4f91dca2548: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40971: connect: Connection refused (error 111)
I20260925 05:17:41.800546 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28542
I20260925 05:17:41.805595 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28180
I20260925 05:17:41.811192 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--webserver_interface=127.13.160.190
--webserver_port=43753
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:41.896449 28704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:41.896724 28704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:41.896764 28704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:41.898022 28704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:41.898101 28704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:41.898128 28704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:41.898166 28704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:41.898214 28704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:41.899887 28704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:41.900213 28704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:41.900460 28704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:41.905601 28712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.905644 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:41.905620 28710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:41.905699 28704 server_base.cc:1034] running on GCE node
I20260925 05:17:41.906215 28704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:41.906533 28704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:41.907701 28704 hybrid_clock.cc:648] HybridClock initialized: now 1790313461907688 us; error 27 us; skew 500 ppm
Sep 25 05:17:41 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313461, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:41.910218 28704 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:41.910772 28704 webserver.cc:460] Webserver started at http://127.13.160.190:43753/ using document root <none> and password file <none>
I20260925 05:17:41.911075 28704 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:41.911151 28704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:41.912583 28704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.913301 28719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:41.913496 28704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:41.913601 28704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "6083a35baf8e4919b1d17843ceda322e"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "eca67cfdd9d0b326b84a9120aea6613c"
server_key_iv: "458f54edd6c38bb415690b45c1d8caf5"
server_key_version: "encryptionkey@0"
I20260925 05:17:41.914008 28704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:41.936587 28704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:41.938617 28704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:41.938793 28704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:41.942929 28704 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33621
I20260925 05:17:41.943001 28771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33621 every 8 connection(s)
I20260925 05:17:41.943429 28704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:41.945065 28772 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:17:41.945720 28772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap starting.
I20260925 05:17:41.946496 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28704
I20260925 05:17:41.946830 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:40971
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=44405
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:41.955402 28772 log.cc:824] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:41.958341 28772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:41.958720 28772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e: Bootstrap complete.
I20260925 05:17:41.961463 28772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.961712 28772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Initialized, Role: FOLLOWER
I20260925 05:17:41.961906 28772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.961998 28772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:41.962065 28772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:41.962124 28772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 2 FOLLOWER]: Advancing to term 3
I20260925 05:17:41.963308 28772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.963413 28772 leader_election.cc:302] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6083a35baf8e4919b1d17843ceda322e; no voters:
I20260925 05:17:41.963655 28772 leader_election.cc:288] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 05:17:41.963733 28776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 3 FOLLOWER]: Leader election won for term 3
I20260925 05:17:41.963879 28776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [term 3 LEADER]: Becoming Leader. State: Replica: 6083a35baf8e4919b1d17843ceda322e, State: Running, Role: LEADER
I20260925 05:17:41.963975 28772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:41.963986 28776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } }
I20260925 05:17:41.964282 28776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6083a35baf8e4919b1d17843ceda322e. Latest consensus state: current_term: 3 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:41.964346 28776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:41.964305 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6083a35baf8e4919b1d17843ceda322e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6083a35baf8e4919b1d17843ceda322e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33621 } } }
I20260925 05:17:41.964546 28781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:41.964578 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6083a35baf8e4919b1d17843ceda322e [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:41.965363 28781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:41.965478 28781 catalog_manager.cc:1259] Loaded cluster ID: f57ca2ccd70d4024ad87cac2d3365b08
I20260925 05:17:41.965512 28781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 05:17:41.965744 28781 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f98c400b9ad google::LogMessage::Fail() at ??:0
@ 0x7f98c400e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f98c400b4de google::LogMessage::Flush() at ??:0
@ 0x7f98c400ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f98cbf2c0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f98cbf358b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f98c5ed90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f98c5ece954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f98c6d326db start_thread at ??:0
@ 0x7f98c2b0961f clone at ??:0
W20260925 05:17:42.046831 28774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:42.047132 28774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:42.047163 28774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:42.048404 28774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:42.048457 28774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:42.048486 28774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:42.050019 28774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:40971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=44405
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:42.050278 28774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:42.050547 28774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:42.055624 28808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:42.055668 28809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:42.055663 28811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:42.056039 28774 server_base.cc:1034] running on GCE node
I20260925 05:17:42.056272 28774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:42.056552 28774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:42.057706 28774 hybrid_clock.cc:648] HybridClock initialized: now 1790313462057692 us; error 36 us; skew 500 ppm
Sep 25 05:17:42 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313462, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:42.060557 28774 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:42.061115 28774 webserver.cc:460] Webserver started at http://127.13.160.129:44405/ using document root <none> and password file <none>
I20260925 05:17:42.061398 28774 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:42.061475 28774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:42.063094 28774 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:42.063916 28818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.064126 28774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:42.064226 28774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "82ea74039083468798a8177f69115cc8"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "5bb24669033a900966b27a6b099d5857"
server_key_iv: "e097f47139108c8b6780468a025a76de"
server_key_version: "encryptionkey@0"
I20260925 05:17:42.064608 28774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:42.087126 28774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:42.089663 28774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:42.089829 28774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:42.090090 28774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:42.090574 28825 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 05:17:42.092430 28774 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 05:17:42.092499 28774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 05:17:42.092545 28774 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 05:17:42.093191 28774 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 05:17:42.093248 28774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:42.093325 28825 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Bootstrap starting.
I20260925 05:17:42.100606 28774 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40971
I20260925 05:17:42.100678 28932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40971 every 8 connection(s)
I20260925 05:17:42.101102 28774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:17:42.102794 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28774
I20260925 05:17:42.103137 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:33817
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=39081
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:17:42.104909 28825 log.cc:824] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:42.107581 28825 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:17:42.107909 28825 tablet_bootstrap.cc:492] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Bootstrap complete.
I20260925 05:17:42.109170 28825 ts_tablet_manager.cc:1397] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260925 05:17:42.111126 28825 raft_consensus.cc:348] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:42.111557 28825 raft_consensus.cc:729] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ea74039083468798a8177f69115cc8, State: Initialized, Role: FOLLOWER
I20260925 05:17:42.111753 28825 consensus_queue.cc:260] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:42.112103 28825 ts_tablet_manager.cc:1428] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260925 05:17:42.189926 28936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:42.190140 28936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:42.190169 28936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:42.191403 28936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:42.191447 28936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:42.191476 28936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:42.192952 28936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:33817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=39081
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:42.193208 28936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:42.193451 28936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:42.198375 28944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:42.198364 28946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:42.198396 28943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:42.198664 28936 server_base.cc:1034] running on GCE node
I20260925 05:17:42.198869 28936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:42.199198 28936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:42.200345 28936 hybrid_clock.cc:648] HybridClock initialized: now 1790313462200330 us; error 27 us; skew 500 ppm
Sep 25 05:17:42 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313462, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:42.203550 28936 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:42.203987 28936 webserver.cc:460] Webserver started at http://127.13.160.130:39081/ using document root <none> and password file <none>
I20260925 05:17:42.204218 28936 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:42.204288 28936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260925 05:17:42.205688 28935 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:17:42.101788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:60107)
0925 05:17:42.101976 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.102021 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:42.102833 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:42.205451 (+102618us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
W20260925 05:17:42.205928 28933 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:33621 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:33621: Client connection negotiation failed: client connection to 127.13.160.190:33621: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 05:17:42.205958 28936 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:42.206265 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.206140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:58943)
0925 05:17:42.206198 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.206242 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 05:17:42.206590 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.206495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:60537)
0925 05:17:42.206534 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.206565 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 05:17:42.206671 28953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260925 05:17:42.206710 28933 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 05:17:42.206853 28936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:17:42.206957 28936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "19c5e28f6e2b4b7fb720dfab80114732"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "83edcb6b10e2235d1bec0321dfc5aeda"
server_key_iv: "6b813c2c857c97a23ce26d013261e667"
server_key_version: "encryptionkey@0"
I20260925 05:17:42.207320 28936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:42.215425 28936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:42.217501 28936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:42.217677 28936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:42.217916 28936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:42.218240 28936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:42.218294 28936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.218335 28936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:42.218364 28936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.224247 28936 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:33817
I20260925 05:17:42.224346 29066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:33817 every 8 connection(s)
I20260925 05:17:42.224674 28936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:17:42.225827 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.225401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:40371)
0925 05:17:42.225564 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.225680 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
W20260925 05:17:42.226255 29067 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:33621 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:33621: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
I20260925 05:17:42.226526 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.226431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:59627)
0925 05:17:42.226475 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.226502 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 05:17:42.226845 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.226772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:59035)
0925 05:17:42.226811 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.226831 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 05:17:42.226982 29067 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 05:17:42.230569 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 28936
I20260925 05:17:42.230820 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:40131
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:33621
--webserver_port=41055
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:42.313212 29070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:42.313416 29070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:42.313439 29070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:42.314611 29070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:42.314653 29070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:42.314683 29070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:42.316229 29070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:40131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=41055
--tserver_master_addrs=127.13.160.190:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:42.316468 29070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:42.316718 29070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:42.321877 29075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:42.321877 29076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:42.321939 29070 server_base.cc:1034] running on GCE node
W20260925 05:17:42.321873 29078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:42.322284 29070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:42.322592 29070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:42.323753 29070 hybrid_clock.cc:648] HybridClock initialized: now 1790313462323730 us; error 37 us; skew 500 ppm
Sep 25 05:17:42 dist-test-slave-9tgz krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313462, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:42.326236 29070 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:42.326716 29070 webserver.cc:460] Webserver started at http://127.13.160.131:41055/ using document root <none> and password file <none>
I20260925 05:17:42.327024 29070 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:42.327096 29070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:42.328362 29070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:42.328943 29085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.329110 29070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:42.329208 29070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "f76e845717d4420fbbb296f678fa2c3b"
format_stamp: "Formatted at 2026-09-25 05:17:40 on dist-test-slave-9tgz"
server_key: "d5e3494b0114c8ff187ba44f44c79743"
server_key_iv: "21336ba476b723be2c1a4fb7b5f0f4f6"
server_key_version: "encryptionkey@0"
I20260925 05:17:42.329563 29070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:42.343760 29070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:42.345713 29070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:42.345880 29070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:42.346127 29070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:42.346437 29070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:42.346491 29070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.346542 29070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:42.346573 29070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:42.352602 29070 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:40131
I20260925 05:17:42.352643 29198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:40131 every 8 connection(s)
I20260925 05:17:42.353006 29070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:17:42.354097 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.353713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:53287)
0925 05:17:42.353873 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.353953 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
W20260925 05:17:42.354521 29199 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:33621 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:33621: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
I20260925 05:17:42.354815 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.354719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:35921)
0925 05:17:42.354768 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.354795 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 05:17:42.355129 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:42.355055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:32791)
0925 05:17:42.355092 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:42.355113 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 05:17:42.355252 29199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 05:17:42.355597 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29070
I20260925 05:17:43.207405 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.207142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:45217)
0925 05:17:43.207297 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:43.207341 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260925 05:17:43.227559 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.227339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:52551)
0925 05:17:43.227472 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:43.227511 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260925 05:17:43.307341 29205 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:43.307514 29205 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:43.307902 29205 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
Sep 25 05:17:43 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313462, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.130@KRBTEST.COM
Sep 25 05:17:43 dist-test-slave-9tgz krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313462, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.131@KRBTEST.COM
I20260925 05:17:43.318370 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.308196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:50935 (local address 127.13.160.130:33817)
0925 05:17:43.308394 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:43.308410 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:43.308443 (+ 33us) server_negotiation.cc:407] Connection header received
0925 05:17:43.308866 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:43.308868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:43.308902 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:43.309056 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:43.309890 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.310795 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.311790 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.312193 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.312315 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:43.314576 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:43.314581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:43.314582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:43.314594 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:43.316578 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317042 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:43.317045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:43.317046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:43.317162 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317432 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:43.317433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:43.317433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:43.317695 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:43.317756 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:43.318127 (+ 371us) server_negotiation.cc:299] Negotiation successful
0925 05:17:43.318250 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":51,"threads_started":1}
I20260925 05:17:43.318357 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.308078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:40131 (local address 127.13.160.129:51421)
0925 05:17:43.308228 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:43.308242 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:43.308313 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:43.309088 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:43.309091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:43.309526 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:43.309707 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:43.309713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.310885 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.310886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:43.311698 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:43.311700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.312287 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.312288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:43.312407 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:43.312992 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:43.312999 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:43.314780 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:43.316833 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:43.316834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:43.316834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:43.317082 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:43.317358 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:43.317359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:43.317445 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:43.317911 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:43.317912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:43.317956 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 05:17:43.318074 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 05:17:43.318262 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":9}
I20260925 05:17:43.318403 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.308213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:51421 (local address 127.13.160.131:40131)
0925 05:17:43.308392 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:43.308402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:43.308425 (+ 23us) server_negotiation.cc:407] Connection header received
0925 05:17:43.308836 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:43.308837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:43.308889 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:43.309032 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:43.309825 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.310794 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.311803 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.312194 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.312314 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:43.314887 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:43.314915 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:43.314915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:43.314927 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:43.316753 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317163 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:43.317165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:43.317166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:43.317289 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317546 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:43.317547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:43.317547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:43.317832 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:43.317922 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:43.318128 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 05:17:43.318283 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":56,"threads_started":1}
I20260925 05:17:43.318658 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.308106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33817 (local address 127.13.160.129:50935)
0925 05:17:43.308240 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:43.308251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:43.308308 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:43.309087 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:43.309090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:43.309527 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:43.309707 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:43.309713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.310957 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.310958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:43.311699 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:43.311701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:43.312290 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:43.312291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:43.312401 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:43.313039 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:43.313049 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:43.314463 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:43.316660 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:43.316667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:43.316669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:43.316964 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:43.317237 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:43.317238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:43.317239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:43.317359 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:43.317783 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:43.317789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:43.317957 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 05:17:43.318074 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 05:17:43.318343 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":197,"thread_start_us":80,"threads_started":1}
W20260925 05:17:43.319410 28821 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
W20260925 05:17:43.319494 28819 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
I20260925 05:17:43.319518 28819 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82ea74039083468798a8177f69115cc8; no voters: 19c5e28f6e2b4b7fb720dfab80114732, f76e845717d4420fbbb296f678fa2c3b
I20260925 05:17:43.319609 29205 raft_consensus.cc:2726] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:17:43.355787 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:43.355573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:46797)
0925 05:17:43.355705 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:43.355750 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
I20260925 05:17:44.208204 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:44.207875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:41259)
0925 05:17:44.208110 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:44.208155 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":82,"threads_started":1}
I20260925 05:17:44.228180 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:44.227978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:38669)
0925 05:17:44.228101 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:44.228138 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260925 05:17:44.356511 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:44.356253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:58647)
0925 05:17:44.356394 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:44.356448 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260925 05:17:45.209014 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:45.208783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:44181)
0925 05:17:45.208928 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:45.208965 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260925 05:17:45.228916 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:45.228678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:36051)
0925 05:17:45.228833 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:45.228871 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260925 05:17:45.357326 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:45.357085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:53119)
0925 05:17:45.357236 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:45.357274 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260925 05:17:45.465101 29216 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:45.465196 29216 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:45.465440 29216 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
W20260925 05:17:45.465878 28819 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
W20260925 05:17:45.465968 28821 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
I20260925 05:17:45.466048 28821 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82ea74039083468798a8177f69115cc8; no voters: 19c5e28f6e2b4b7fb720dfab80114732, f76e845717d4420fbbb296f678fa2c3b
I20260925 05:17:45.466132 29216 raft_consensus.cc:2726] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:17:46.209793 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:46.209576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:35863)
0925 05:17:46.209705 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:46.209743 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260925 05:17:46.229588 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:46.229355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:47311)
0925 05:17:46.229504 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:46.229542 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260925 05:17:46.358052 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:46.357822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:44577)
0925 05:17:46.357965 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:46.358009 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260925 05:17:47.210701 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:47.210440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:36711)
0925 05:17:47.210604 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:47.210653 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260925 05:17:47.230305 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:47.230076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:47131)
0925 05:17:47.230216 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:47.230260 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260925 05:17:47.358836 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:47.358568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:52607)
0925 05:17:47.358722 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:47.358771 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260925 05:17:48.211519 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:48.211284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:38755)
0925 05:17:48.211419 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:48.211466 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260925 05:17:48.231130 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:48.230836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:51175)
0925 05:17:48.231038 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:48.231084 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":138,"threads_started":1}
I20260925 05:17:48.359696 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:48.359447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:58081)
0925 05:17:48.359597 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:48.359646 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260925 05:17:48.367115 29226 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:48.367240 29226 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:48.367435 29226 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
W20260925 05:17:48.367868 28819 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
W20260925 05:17:48.367935 28821 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
I20260925 05:17:48.367967 28821 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82ea74039083468798a8177f69115cc8; no voters: 19c5e28f6e2b4b7fb720dfab80114732, f76e845717d4420fbbb296f678fa2c3b
I20260925 05:17:48.368069 29226 raft_consensus.cc:2726] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:17:49.212296 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:49.212072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:41937)
0925 05:17:49.212206 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:49.212249 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260925 05:17:49.231834 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:49.231606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:39531)
0925 05:17:49.231741 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:49.231788 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260925 05:17:49.360308 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:49.360122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:60113)
0925 05:17:49.360236 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:49.360267 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260925 05:17:50.212970 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:50.212730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:57809)
0925 05:17:50.212876 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:50.212920 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260925 05:17:50.232613 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:50.232350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:56811)
0925 05:17:50.232532 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:50.232568 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":137,"threads_started":1}
I20260925 05:17:50.361061 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:50.360784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:35887)
0925 05:17:50.360956 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:50.361011 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
I20260925 05:17:51.213856 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:51.213559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:59501)
0925 05:17:51.213755 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:51.213806 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
I20260925 05:17:51.233464 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:51.233200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:43437)
0925 05:17:51.233362 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:51.233417 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":128,"threads_started":1}
I20260925 05:17:51.361786 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:51.361578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:44691)
0925 05:17:51.361696 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:51.361740 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260925 05:17:51.837033 29236 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:51.837173 29236 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:51.837406 29236 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
W20260925 05:17:51.837815 28821 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
W20260925 05:17:51.837953 28819 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
I20260925 05:17:51.837996 28819 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82ea74039083468798a8177f69115cc8; no voters: 19c5e28f6e2b4b7fb720dfab80114732, f76e845717d4420fbbb296f678fa2c3b
I20260925 05:17:51.838094 29236 raft_consensus.cc:2726] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:17:52.214715 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:52.214496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:43887)
0925 05:17:52.214630 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:52.214667 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260925 05:17:52.234289 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:52.234065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:58325)
0925 05:17:52.234209 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:52.234243 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260925 05:17:52.362447 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:52.362220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:45433)
0925 05:17:52.362363 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:52.362403 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260925 05:17:53.215466 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:53.215236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:43365)
0925 05:17:53.215366 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:53.215414 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260925 05:17:53.234994 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:53.234736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:58311)
0925 05:17:53.234864 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:53.234945 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260925 05:17:53.363252 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:53.363010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:36567)
0925 05:17:53.363161 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:53.363204 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260925 05:17:54.216289 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:54.216055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:50891)
0925 05:17:54.216197 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:54.216241 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260925 05:17:54.235669 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:54.235441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:42535)
0925 05:17:54.235581 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:54.235626 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260925 05:17:54.363936 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:54.363691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:46173)
0925 05:17:54.363839 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:54.363887 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260925 05:17:55.217129 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:55.216870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:50549)
0925 05:17:55.217029 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:55.217078 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260925 05:17:55.236416 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:55.236199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:33067)
0925 05:17:55.236328 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:55.236370 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260925 05:17:55.364760 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:55.364513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:33989)
0925 05:17:55.364662 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:55.364711 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260925 05:17:56.090313 29249 raft_consensus.cc:482] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:56.090417 29249 raft_consensus.cc:504] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76e845717d4420fbbb296f678fa2c3b" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 40131 } } peers { permanent_uuid: "19c5e28f6e2b4b7fb720dfab80114732" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33817 } } peers { permanent_uuid: "82ea74039083468798a8177f69115cc8" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40971 } }
I20260925 05:17:56.090603 29249 leader_election.cc:288] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131), 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817)
W20260925 05:17:56.091027 28821 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76e845717d4420fbbb296f678fa2c3b (127.13.160.131:40131): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
W20260925 05:17:56.091101 28819 leader_election.cc:341] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19c5e28f6e2b4b7fb720dfab80114732 (127.13.160.130:33817): Not found: Tablet not found: c753c56b0df74badadaec4f91dca2548
I20260925 05:17:56.091179 28819 leader_election.cc:302] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82ea74039083468798a8177f69115cc8; no voters: 19c5e28f6e2b4b7fb720dfab80114732, f76e845717d4420fbbb296f678fa2c3b
I20260925 05:17:56.091262 29249 raft_consensus.cc:2726] T c753c56b0df74badadaec4f91dca2548 P 82ea74039083468798a8177f69115cc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:17:56.217921 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:56.217678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:33507)
0925 05:17:56.217823 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:56.217872 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260925 05:17:56.237196 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:56.236952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:39173)
0925 05:17:56.237094 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:56.237133 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260925 05:17:56.365571 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:56.365355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:49439)
0925 05:17:56.365482 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:56.365522 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260925 05:17:57.218843 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:57.218516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.129:34867)
0925 05:17:57.218743 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:57.218794 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":60,"threads_started":1}
I20260925 05:17:57.238022 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:57.237809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.130:50655)
0925 05:17:57.237942 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:57.237978 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260925 05:17:57.359778 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28774
I20260925 05:17:57.364742 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 28936
I20260925 05:17:57.366528 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:57.366321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33621 (local address 127.13.160.131:46185)
0925 05:17:57.366449 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:57.366486 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:33621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260925 05:17:57.369248 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29070
2026-09-25T05:17:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18232 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:17:57 dist-test-slave-9tgz krb5kdc[29258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:17:57 dist-test-slave-9tgz krb5kdc[29258](info): set up 2 sockets
Sep 25 05:17:57 dist-test-slave-9tgz krb5kdc[29258](info): commencing operation
krb5kdc: starting...
W20260925 05:17:58.403808 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:17:58 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:17:58.520123 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:58.603514 29271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:58.603727 29271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:58.603749 29271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:58.605019 29271 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 05:17:58.605063 29271 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 05:17:58.605077 29271 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 05:17:58.605088 29271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:17:58.605103 29271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:58.605151 29271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:58.605168 29271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:17:58.605190 29271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:17:58.606731 29271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:58.607020 29271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:58.607270 29271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:17:58.612325 29271 server_base.cc:1034] running on GCE node
W20260925 05:17:58.612282 29277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:58.612314 29276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:58.612308 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:58.612797 29271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:58.613117 29271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:58.614285 29271 hybrid_clock.cc:648] HybridClock initialized: now 1790313478614281 us; error 38 us; skew 500 ppm
Sep 25 05:17:58 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313478, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:58.621938 29271 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:17:58.622428 29271 webserver.cc:460] Webserver started at http://127.13.160.190:42533/ using document root <none> and password file <none>
I20260925 05:17:58.622685 29271 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:58.622756 29271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:58.622946 29271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:58.624056 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "63e63f05d8df6aa8119b4a68139f6733"
server_key_iv: "ad6361ede5b1372b344b12a412970c27"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.624420 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "63e63f05d8df6aa8119b4a68139f6733"
server_key_iv: "ad6361ede5b1372b344b12a412970c27"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.625856 29271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:17:58.626608 29286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:58.626809 29271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:17:58.626935 29271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "63e63f05d8df6aa8119b4a68139f6733"
server_key_iv: "ad6361ede5b1372b344b12a412970c27"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.627024 29271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:58.642452 29271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:58.644501 29271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:58.644682 29271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:58.648917 29271 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:43273
I20260925 05:17:58.648967 29338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:43273 every 8 connection(s)
I20260925 05:17:58.649358 29271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:17:58.650235 29339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:58.652689 29339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap starting.
I20260925 05:17:58.653321 29339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:58.653632 29339 log.cc:824] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:58.654321 29339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: No bootstrap required, opened a new log
I20260925 05:17:58.655232 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29271
I20260925 05:17:58.655331 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:17:58.655524 13954 external_mini_cluster.cc:1428] Setting key 49cc152ff2f540823bb1604239b54d19
I20260925 05:17:58.656543 29339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:17:58.656723 29339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:58.656765 29339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Initialized, Role: FOLLOWER
I20260925 05:17:58.656909 29339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:17:58.656999 29339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:17:58.657032 29339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:17:58.657086 29339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:58.657759 29339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:17:58.657886 29339 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c4a8ec1b9204cb6baa2ce868e8db9e8; no voters:
I20260925 05:17:58.658085 29339 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:17:58.658121 29344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:58.658221 29344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Running, Role: LEADER
I20260925 05:17:58.658325 29344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:17:58.658358 29339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:17:58.659399 29345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:17:58.659409 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4a8ec1b9204cb6baa2ce868e8db9e8. Latest consensus state: current_term: 1 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:17:58.659503 29345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:17:58.659515 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
W20260925 05:17:58.660465 29359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:17:58.660521 29359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:17:58.660570 29354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:17:58.660754 29354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:17:58.663101 29354 catalog_manager.cc:1347] Generated new cluster ID: 79e3d3b7c5544eddae77762ab95f81db
I20260925 05:17:58.663153 29354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:17:58 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:58.667055 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:58.656087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.13.160.190:43273)
0925 05:17:58.656263 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:58.656278 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:58.656304 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:17:58.656518 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:58.656524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:58.656573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:58.656681 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:58.657481 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.659076 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.660238 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.660819 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.660925 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:58.663688 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:58.663692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:58.663693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:58.663705 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:58.665458 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:58.665856 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:58.665859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:58.665860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:58.665983 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:58.666197 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:58.666198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:58.666199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:58.666441 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:58.666622 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:58.666694 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 05:17:58.666834 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260925 05:17:58.677892 29354 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:17:58.678571 29354 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:17:58.683555 29354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Generated new TSK 0
I20260925 05:17:58.683722 29354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:17:58.714236 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:58.831696 29365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:58.832000 29365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:58.832043 29365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:58.833760 29365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:58.833833 29365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:58.833875 29365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:17:58.836284 29365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:58.836561 29365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:58.836834 29365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:58.842106 29370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:58.842103 29373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:58.842118 29371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:58.842330 29365 server_base.cc:1034] running on GCE node
I20260925 05:17:58.842532 29365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:58.842800 29365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:58.843971 29365 hybrid_clock.cc:648] HybridClock initialized: now 1790313478843947 us; error 39 us; skew 500 ppm
Sep 25 05:17:58 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313478, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:58.846791 29365 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:17:58.847247 29365 webserver.cc:460] Webserver started at http://127.13.160.129:34089/ using document root <none> and password file <none>
I20260925 05:17:58.847496 29365 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:58.847543 29365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:58.847711 29365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:58.848769 29365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "337603697ab8457fbdc4998795542193"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "65d8c663dae0adcec445762981d437c3"
server_key_iv: "c17d69e1c98c03b89e72fd22508e1db5"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.849129 29365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "337603697ab8457fbdc4998795542193"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "65d8c663dae0adcec445762981d437c3"
server_key_iv: "c17d69e1c98c03b89e72fd22508e1db5"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.850443 29365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:17:58.851233 29380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:58.851406 29365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:17:58.851521 29365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "337603697ab8457fbdc4998795542193"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "65d8c663dae0adcec445762981d437c3"
server_key_iv: "c17d69e1c98c03b89e72fd22508e1db5"
server_key_version: "encryptionkey@0"
I20260925 05:17:58.851599 29365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:58.892796 29365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:58.895071 29365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:58.895242 29365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:58.895514 29365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:58.895856 29365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:58.895912 29365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:58.895947 29365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:58.895977 29365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:58.901479 29365 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:33465
I20260925 05:17:58.901583 29493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:33465 every 8 connection(s)
I20260925 05:17:58.901894 29365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:17:58 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313478, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:58.910616 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29365
I20260925 05:17:58.910729 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:17:58.910996 13954 external_mini_cluster.cc:1428] Setting key 4ff2ec49f0ca87e4ee6f5c03abfe1de9
I20260925 05:17:58.911836 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:58.902647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:40605)
0925 05:17:58.902809 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:58.902853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:58.903405 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:58.903675 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:58.903679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:58.904270 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:58.904466 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:58.904471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.905198 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.905200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:58.906029 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:58.906031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.906459 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.906460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:58.906601 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:58.907230 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:58.907238 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:58.908660 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:58.910081 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:58.910088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:58.910089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:58.910303 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:58.910485 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:58.910486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:58.910487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:58.910642 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:58.911063 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:58.911071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:58.911241 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 05:17:58.911386 (+ 145us) client_negotiation.cc:240] Negotiation successful
0925 05:17:58.911589 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260925 05:17:58.911849 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:58.902681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:40605 (local address 127.13.160.190:43273)
0925 05:17:58.902742 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:58.902744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:58.902975 (+ 231us) server_negotiation.cc:407] Connection header received
0925 05:17:58.903516 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:58.903517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:58.903532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:58.903586 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:58.904572 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.905095 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.906130 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:58.906360 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:58.906447 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:58.908754 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:58.908755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:58.908755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:58.908764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:58.910001 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:58.910369 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:58.910369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:58.910370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:58.910417 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:58.910768 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:58.910769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:58.910770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:58.910995 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:58.911068 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:58.911654 (+ 586us) server_negotiation.cc:299] Negotiation successful
0925 05:17:58.911744 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:17:58.912482 29494 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:17:58.912580 29494 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:58.912778 29494 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:17:58.913368 29303 ts_manager.cc:194] Registered new tserver with Master: 337603697ab8457fbdc4998795542193 (127.13.160.129:33465)
I20260925 05:17:58.914326 29303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:40605
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:17:58.935670 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:59.018158 29499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:59.018368 29499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:59.018391 29499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:59.019492 29499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:59.019527 29499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:59.019553 29499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:17:59.020982 29499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:59.021178 29499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:59.021346 29499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:59.026232 29505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:59.026219 29507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:59.026219 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:59.026681 29499 server_base.cc:1034] running on GCE node
I20260925 05:17:59.026955 29499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:59.027227 29499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:59.028369 29499 hybrid_clock.cc:648] HybridClock initialized: now 1790313479028348 us; error 36 us; skew 500 ppm
Sep 25 05:17:59 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313479, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:59.031057 29499 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:17:59.031615 29499 webserver.cc:460] Webserver started at http://127.13.160.130:35561/ using document root <none> and password file <none>
I20260925 05:17:59.031965 29499 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:59.032038 29499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:59.032186 29499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:59.033277 29499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "2c2795a5bf424ec9845e89ee5a28263b"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "1b1bf379794fc3d073ba66e43f454dee"
server_key_iv: "f6742c6fd934d69fb7ed21e7123b5197"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.033632 29499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "2c2795a5bf424ec9845e89ee5a28263b"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "1b1bf379794fc3d073ba66e43f454dee"
server_key_iv: "f6742c6fd934d69fb7ed21e7123b5197"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.035004 29499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 05:17:59.035713 29514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.035872 29499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:17:59.035976 29499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "2c2795a5bf424ec9845e89ee5a28263b"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "1b1bf379794fc3d073ba66e43f454dee"
server_key_iv: "f6742c6fd934d69fb7ed21e7123b5197"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.036054 29499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:59.064563 29499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:59.066628 29499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:59.066800 29499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:59.067090 29499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:59.067407 29499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:59.067463 29499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.067510 29499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:59.067534 29499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.073206 29499 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:42113
I20260925 05:17:59.073302 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:42113 every 8 connection(s)
I20260925 05:17:59.073616 29499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:17:59 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313479, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:59.080752 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29499
I20260925 05:17:59.080860 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:17:59.081079 13954 external_mini_cluster.cc:1428] Setting key 3131d9535365e9fa59904cce156f67c4
I20260925 05:17:59.083455 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.074478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:46013 (local address 127.13.160.190:43273)
0925 05:17:59.074531 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.074533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.074769 (+ 236us) server_negotiation.cc:407] Connection header received
0925 05:17:59.075377 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.075379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.075390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.075444 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:59.076435 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.076937 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.077979 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.078181 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.078268 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.080435 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:59.080437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:59.080438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:59.080447 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:59.082054 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.082459 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.082461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.082461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.082524 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.082778 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.082779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.082780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.082937 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:59.082999 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.083271 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.083364 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:17:59.083665 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.074457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:46013)
0925 05:17:59.074638 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.074674 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.075266 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.075532 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.075536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.076131 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:59.076329 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.076334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.077064 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.077065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.077882 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.077885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.078274 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.078276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.078408 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.078993 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:59.079001 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:59.080332 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:59.082154 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:59.082161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:59.082163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:59.082384 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:59.082589 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:59.082589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:59.082590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:59.082702 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:59.082989 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:59.082996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:59.083139 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.083251 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.083409 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260925 05:17:59.084120 29628 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:17:59.084211 29628 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:59.084405 29628 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:17:59.084805 29303 ts_manager.cc:194] Registered new tserver with Master: 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113)
I20260925 05:17:59.085348 29303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:46013
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:17:59.106076 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:17:59.188036 29633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:17:59.188230 29633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:17:59.188253 29633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:17:59.189304 29633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:17:59.189337 29633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:17:59.189363 29633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:17:59.190824 29633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:17:59.191082 29633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:17:59.191303 29633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:17:59.196262 29639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:17:59.196264 29638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:59.196350 29633 server_base.cc:1034] running on GCE node
W20260925 05:17:59.196264 29641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:17:59.196760 29633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:17:59.197002 29633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:17:59.198143 29633 hybrid_clock.cc:648] HybridClock initialized: now 1790313479198131 us; error 28 us; skew 500 ppm
Sep 25 05:17:59 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313479, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:17:59.200709 29633 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:17:59.201159 29633 webserver.cc:460] Webserver started at http://127.13.160.131:37853/ using document root <none> and password file <none>
I20260925 05:17:59.201402 29633 fs_manager.cc:359] Metadata directory not provided
I20260925 05:17:59.201470 29633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:17:59.201632 29633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:17:59.202665 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "e86157afa28545e2a932c03d3dd6cb68"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "78af29e4de66a08fc4608307b1173926"
server_key_iv: "0ad457fdccc52b779ad8360b9ba4838c"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.203061 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "e86157afa28545e2a932c03d3dd6cb68"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "78af29e4de66a08fc4608307b1173926"
server_key_iv: "0ad457fdccc52b779ad8360b9ba4838c"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.204367 29633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:17:59.205084 29648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.205262 29633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:17:59.205361 29633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "e86157afa28545e2a932c03d3dd6cb68"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "78af29e4de66a08fc4608307b1173926"
server_key_iv: "0ad457fdccc52b779ad8360b9ba4838c"
server_key_version: "encryptionkey@0"
I20260925 05:17:59.205441 29633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:17:59.222963 29633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:17:59.224881 29633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:17:59.225041 29633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:17:59.225275 29633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:17:59.225585 29633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:17:59.225638 29633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.225693 29633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:17:59.225721 29633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:17:59.231777 29633 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42753
I20260925 05:17:59.231819 29761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42753 every 8 connection(s)
I20260925 05:17:59.232182 29633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:17:59 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313479, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:17:59.240864 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29633
I20260925 05:17:59.240967 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:17:59.241153 13954 external_mini_cluster.cc:1428] Setting key 528503cef44c8aa5ee4aa92d9b3d130c
I20260925 05:17:59.241420 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.232916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:33683 (local address 127.13.160.190:43273)
0925 05:17:59.232960 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.232962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.233158 (+ 196us) server_negotiation.cc:407] Connection header received
0925 05:17:59.233653 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.233654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.233662 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.233722 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:59.234633 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.235196 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.236144 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.236323 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.236373 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.238454 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:59.238455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:59.238456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:59.238462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:59.239778 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.240237 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.240238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.240238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.240291 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.240634 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.240635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.240635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.240741 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:59.240797 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.241210 (+ 413us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.241297 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:17:59.241607 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.232878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:33683)
0925 05:17:59.233046 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.233088 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.233570 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.233793 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.233796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.234355 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:17:59.234557 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.234562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.235272 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.235273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.236070 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.236073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.236393 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.236393 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.236503 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.237037 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:17:59.237045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:17:59.238365 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:17:59.239847 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:59.239887 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:59.239890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:59.240171 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:59.240436 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:17:59.240437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:17:59.240437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:17:59.240549 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:17:59.240832 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:17:59.240840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:17:59.241033 (+ 193us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.241159 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.241333 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260925 05:17:59.242173 29762 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:17:59.242291 29762 heartbeater.cc:464] Registering TS with master...
I20260925 05:17:59.242467 29762 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:17:59.242882 29302 ts_manager.cc:194] Registered new tserver with Master: e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
I20260925 05:17:59.243323 29302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:33683
I20260925 05:17:59.253232 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:17:59.260598 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.254592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.13.160.190:43273)
0925 05:17:59.254658 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.254660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.254832 (+ 172us) server_negotiation.cc:407] Connection header received
0925 05:17:59.254959 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.254959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.254970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.255031 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:17:59.255570 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.256049 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.256735 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.256945 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.256997 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.258181 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:17:59.258182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:17:59.258182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:17:59.258187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:17:59.259623 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.259918 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.259919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.259920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.259987 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:17:59.260163 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:17:59.260164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:17:59.260164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:17:59.260322 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:17:59.260397 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.260479 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.260534 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:17:59.262269 29302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:17:59.263940 29302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:17:59.270654 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:42113 (local address 127.0.0.1:52612)
0925 05:17:59.266623 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.266632 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.266672 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.267270 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.267271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.267273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.267312 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.267314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268540 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.268541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.269480 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.269482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.270363 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.270364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.270432 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.270444 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.270509 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.270548 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260925 05:17:59.270682 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.13.160.130:42113)
0925 05:17:59.267041 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.267045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.267055 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:59.267094 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.267096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.267113 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.267187 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.267386 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268426 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.269575 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.270280 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.270360 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.270420 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.270547 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.270597 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":90,"threads_started":1}
I20260925 05:17:59.270859 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42753 (local address 127.0.0.1:58054)
0925 05:17:59.266482 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.266532 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.266600 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.267294 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.267295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.267297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.267347 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.267348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268275 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.268276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.269677 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.269680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.270614 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.270615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.270672 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.270676 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.270731 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.270780 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":39,"thread_start_us":44,"threads_started":1}
I20260925 05:17:59.271081 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.13.160.131:42753)
0925 05:17:59.267058 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.267063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.267073 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:59.267114 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.267116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.267132 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.267207 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.267434 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268171 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.269782 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.270528 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.270603 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.270657 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.270949 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.271000 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":88,"threads_started":1}
I20260925 05:17:59.271821 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 (DEFAULT_TABLE table=test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:59.271845 29696 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 (DEFAULT_TABLE table=test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:59.272122 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1b6f92abc470588d49b8ecd4907e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:59.272184 29696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1b6f92abc470588d49b8ecd4907e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:59.274677 29779 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Bootstrap starting.
I20260925 05:17:59.274722 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.0.0.1:41706)
0925 05:17:59.266724 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.266732 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.266772 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.266963 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.266966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.266975 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.267057 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.267060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268374 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.268375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.269677 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.269680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.274438 (+ 4758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.274440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.274522 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.274528 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.274583 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.274629 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":80,"spinlock_wait_cycles":384,"thread_start_us":101,"threads_started":1}
I20260925 05:17:59.274991 29780 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Bootstrap starting.
I20260925 05:17:59.275132 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.266564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.13.160.129:33465)
0925 05:17:59.266763 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.266767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.266777 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:17:59.266819 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.266821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.266834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.266947 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.267149 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.268271 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.270696 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.271439 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.271502 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.274957 (+ 3455us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.274998 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.275048 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260925 05:17:59.275604 29779 tablet_bootstrap.cc:654] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:59.275625 29780 tablet_bootstrap.cc:654] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:59.275820 29428 tablet_service.cc:1467] Processing CreateTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 (DEFAULT_TABLE table=test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:17:59.276170 29780 log.cc:824] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:59.276193 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e1b6f92abc470588d49b8ecd4907e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:17:59.276207 29779 log.cc:824] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:59.276876 29780 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: No bootstrap required, opened a new log
I20260925 05:17:59.276925 29779 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: No bootstrap required, opened a new log
I20260925 05:17:59.276957 29780 ts_tablet_manager.cc:1397] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:59.276993 29779 ts_tablet_manager.cc:1397] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:59.278344 29780 raft_consensus.cc:348] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.278484 29780 raft_consensus.cc:374] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:59.278506 29780 raft_consensus.cc:729] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2795a5bf424ec9845e89ee5a28263b, State: Initialized, Role: FOLLOWER
I20260925 05:17:59.278625 29780 consensus_queue.cc:260] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.278896 29780 ts_tablet_manager.cc:1428] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:59.278949 29628 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
I20260925 05:17:59.278971 29779 raft_consensus.cc:348] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.279121 29779 raft_consensus.cc:374] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:59.279151 29779 raft_consensus.cc:729] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86157afa28545e2a932c03d3dd6cb68, State: Initialized, Role: FOLLOWER
I20260925 05:17:59.279255 29779 consensus_queue.cc:260] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.279544 29779 ts_tablet_manager.cc:1428] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 05:17:59.279685 29762 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
I20260925 05:17:59.279992 29785 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Bootstrap starting.
I20260925 05:17:59.280721 29785 tablet_bootstrap.cc:654] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Neither blocks nor log segments found. Creating new log.
I20260925 05:17:59.281212 29785 log.cc:824] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Log is configured to *not* fsync() on all Append() calls
I20260925 05:17:59.281841 29785 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: No bootstrap required, opened a new log
I20260925 05:17:59.281893 29785 ts_tablet_manager.cc:1397] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:17:59.283296 29785 raft_consensus.cc:348] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.283427 29785 raft_consensus.cc:374] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:17:59.283479 29785 raft_consensus.cc:729] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337603697ab8457fbdc4998795542193, State: Initialized, Role: FOLLOWER
I20260925 05:17:59.283619 29785 consensus_queue.cc:260] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.283861 29785 ts_tablet_manager.cc:1428] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:17:59.283912 29494 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
W20260925 05:17:59.324183 29629 tablet.cc:2369] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:17:59.402635 29495 tablet.cc:2369] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:17:59.482815 29763 tablet.cc:2369] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:17:59.592558 29784 raft_consensus.cc:482] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:59.592741 29784 raft_consensus.cc:504] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.593186 29784 leader_election.cc:288] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337603697ab8457fbdc4998795542193 (127.13.160.129:33465), 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113)
I20260925 05:17:59.594624 29787 raft_consensus.cc:482] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:17:59.594765 29787 raft_consensus.cc:504] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.595412 29787 leader_election.cc:288] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753), 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113)
I20260925 05:17:59.596797 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.593403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:42113 (local address 127.13.160.131:36645)
0925 05:17:59.593463 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.593486 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.593535 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.593711 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.593712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.593713 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.593789 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.593791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.594450 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.594452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.595747 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.595750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596538 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.596630 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.596632 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.596675 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.596723 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":14}
I20260925 05:17:59.596822 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.593451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36607 (local address 127.13.160.129:33465)
0925 05:17:59.593531 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.593534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.593682 (+ 148us) server_negotiation.cc:407] Connection header received
0925 05:17:59.593754 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.593756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.593765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.593840 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.594007 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.594500 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.595861 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596418 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596468 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.596478 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.596719 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.596762 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 05:17:59.597057 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.593429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.13.160.131:36607)
0925 05:17:59.593600 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.593618 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.593697 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.593841 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.593842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.593843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.593926 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.593928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.594580 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.594581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.595746 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.595750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596538 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.596623 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.596627 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.596924 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.596992 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":9,"thread_start_us":118,"threads_started":1}
I20260925 05:17:59.597431 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "e86157afa28545e2a932c03d3dd6cb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337603697ab8457fbdc4998795542193" is_pre_election: true
I20260925 05:17:59.597501 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.593530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36645 (local address 127.13.160.130:42113)
0925 05:17:59.593577 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.593579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.593588 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:17:59.593627 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.593627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.593635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.593703 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.593862 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.594357 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.595855 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596418 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596497 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.596506 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.597374 (+ 868us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.597423 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:17:59.597551 29448 raft_consensus.cc:2454] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86157afa28545e2a932c03d3dd6cb68 in term 0.
I20260925 05:17:59.597765 29651 leader_election.cc:302] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337603697ab8457fbdc4998795542193, e86157afa28545e2a932c03d3dd6cb68; no voters:
I20260925 05:17:59.597765 29582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "e86157afa28545e2a932c03d3dd6cb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2795a5bf424ec9845e89ee5a28263b" is_pre_election: true
I20260925 05:17:59.597911 29582 raft_consensus.cc:2454] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86157afa28545e2a932c03d3dd6cb68 in term 0.
I20260925 05:17:59.598006 29784 raft_consensus.cc:2781] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:17:59.598086 29784 raft_consensus.cc:482] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:17:59.598122 29784 raft_consensus.cc:3037] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:59.599018 29784 raft_consensus.cc:504] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.599166 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.595281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:42113 (local address 127.13.160.129:59837)
0925 05:17:59.595571 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.595579 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.595619 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.595972 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.595973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.595975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.596034 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.596036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596912 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.598065 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.598067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.598834 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.598957 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.598963 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.599031 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.599078 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":5,"thread_start_us":53,"threads_started":1}
I20260925 05:17:59.599176 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.595715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:59837 (local address 127.13.160.130:42113)
0925 05:17:59.595827 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.595828 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.595836 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:17:59.595859 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.595860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.595869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.596096 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.596129 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596823 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598169 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.598730 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598800 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.598808 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.599048 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.599100 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260925 05:17:59.599210 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.595434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:57135 (local address 127.13.160.131:42753)
0925 05:17:59.595476 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.595478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.595590 (+ 112us) server_negotiation.cc:407] Connection header received
0925 05:17:59.595690 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.595691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.595700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.595729 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:17:59.595957 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596687 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598172 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.598710 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598760 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.598770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.599104 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.599145 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:17:59.599370 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "e86157afa28545e2a932c03d3dd6cb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337603697ab8457fbdc4998795542193"
I20260925 05:17:59.599398 29582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "337603697ab8457fbdc4998795542193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2795a5bf424ec9845e89ee5a28263b" is_pre_election: true
I20260925 05:17:59.599472 29582 raft_consensus.cc:2454] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337603697ab8457fbdc4998795542193 in term 0.
I20260925 05:17:59.599496 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.595280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42753 (local address 127.13.160.129:57135)
0925 05:17:59.595522 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.595534 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:17:59.595617 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:17:59.595792 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:17:59.595792 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:17:59.595794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:17:59.595874 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.595876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.596779 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.596779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.598065 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:17:59.598067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.598832 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.598833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:17:59.598996 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.599001 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:17:59.599319 (+ 318us) client_negotiation.cc:240] Negotiation successful
0925 05:17:59.599373 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":5,"thread_start_us":56,"threads_started":1}
I20260925 05:17:59.599689 29381 leader_election.cc:302] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2795a5bf424ec9845e89ee5a28263b, 337603697ab8457fbdc4998795542193; no voters:
I20260925 05:17:59.599474 29448 raft_consensus.cc:3037] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:59.599754 29582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "e86157afa28545e2a932c03d3dd6cb68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2795a5bf424ec9845e89ee5a28263b"
I20260925 05:17:59.599817 29582 raft_consensus.cc:3037] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:17:59.599905 29716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e1b6f92abc470588d49b8ecd4907e8" candidate_uuid: "337603697ab8457fbdc4998795542193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86157afa28545e2a932c03d3dd6cb68" is_pre_election: true
I20260925 05:17:59.600020 29716 raft_consensus.cc:2379] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 337603697ab8457fbdc4998795542193 in current term 1: Already voted for candidate e86157afa28545e2a932c03d3dd6cb68 in this term.
I20260925 05:17:59.600244 29784 leader_election.cc:288] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [CANDIDATE]: Term 1 election: Requested vote from peers 337603697ab8457fbdc4998795542193 (127.13.160.129:33465), 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113)
I20260925 05:17:59.600646 29448 raft_consensus.cc:2454] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86157afa28545e2a932c03d3dd6cb68 in term 1.
I20260925 05:17:59.600661 29582 raft_consensus.cc:2454] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86157afa28545e2a932c03d3dd6cb68 in term 1.
I20260925 05:17:59.600749 29787 raft_consensus.cc:2741] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 05:17:59.600874 29649 leader_election.cc:302] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2795a5bf424ec9845e89ee5a28263b, e86157afa28545e2a932c03d3dd6cb68; no voters:
I20260925 05:17:59.600966 29784 raft_consensus.cc:2781] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:17:59.601130 29784 raft_consensus.cc:686] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 LEADER]: Becoming Leader. State: Replica: e86157afa28545e2a932c03d3dd6cb68, State: Running, Role: LEADER
I20260925 05:17:59.601253 29784 consensus_queue.cc:237] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:17:59.602169 29302 catalog_manager.cc:5495] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 reported cstate change: term changed from 0 to 1, leader changed from <none> to e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "e86157afa28545e2a932c03d3dd6cb68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } health_report { overall_health: UNKNOWN } } }
I20260925 05:17:59.625737 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.623172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.13.160.131:42753)
0925 05:17:59.623219 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:17:59.623222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:17:59.623242 (+ 20us) server_negotiation.cc:407] Connection header received
0925 05:17:59.623285 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:17:59.623286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:17:59.623307 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:17:59.623393 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:17:59.623507 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.624058 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.624839 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:17:59.625061 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:17:59.625141 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:17:59.625316 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:17:59.625454 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:17:59.625559 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 05:17:59.625578 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 05:17:59.625600 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:17:59.627710 29448 raft_consensus.cc:1261] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Refusing update from remote peer e86157afa28545e2a932c03d3dd6cb68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:59.627743 29582 raft_consensus.cc:1261] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Refusing update from remote peer e86157afa28545e2a932c03d3dd6cb68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:17:59.628090 29792 consensus_queue.cc:1035] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:59.628228 29784 consensus_queue.cc:1035] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:17:59.630740 29795 mvcc.cc:205] Tried to move back new op lower bound from 7333124012552183808 to 7333124012447358976. Current Snapshot: MvccSnapshot[applied={T|T < 7333124012552183808}]
I20260925 05:17:59.631377 29797 mvcc.cc:205] Tried to move back new op lower bound from 7333124012552183808 to 7333124012447358976. Current Snapshot: MvccSnapshot[applied={T|T < 7333124012552183808 or (T in {7333124012552183808})}]
I20260925 05:17:59.632166 29796 mvcc.cc:205] Tried to move back new op lower bound from 7333124012552183808 to 7333124012447358976. Current Snapshot: MvccSnapshot[applied={T|T < 7333124012552183808}]
I20260925 05:17:59.633303 29302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:59.633409 29302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:17:59.634213 29302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Sending DeleteTablet for 3 replicas of tablet 62e1b6f92abc470588d49b8ecd4907e8
I20260925 05:17:59.634522 29696 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:59 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:58054
I20260925 05:17:59.634588 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:59 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:52612
I20260925 05:17:59.634653 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29365
I20260925 05:17:59.634761 29428 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:59 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:41706
I20260925 05:17:59.635402 29805 tablet_replica.cc:330] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: stopping tablet replica
I20260925 05:17:59.635531 29805 raft_consensus.cc:2229] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:17:59.635643 29805 raft_consensus.cc:2258] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:59.636111 29805 ts_tablet_manager.cc:1905] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:59.638338 29805 ts_tablet_manager.cc:1918] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:59.638427 29805 log.cc:1196] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/62e1b6f92abc470588d49b8ecd4907e8
I20260925 05:17:59.638752 29805 ts_tablet_manager.cc:1939] T 62e1b6f92abc470588d49b8ecd4907e8 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting consensus metadata
I20260925 05:17:59.639240 29287 catalog_manager.cc:4841] TS 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): tablet 62e1b6f92abc470588d49b8ecd4907e8 (table test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]) successfully deleted
W20260925 05:17:59.641733 29289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:33465 (error 108)
I20260925 05:17:59.641990 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29499
I20260925 05:17:59.642639 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:17:59.642162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.0.0.1:41708)
0925 05:17:59.642568 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0925 05:17:59.642604 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:33465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387}
I20260925 05:17:59.642743 29804 tablet_replica.cc:330] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: stopping tablet replica
I20260925 05:17:59.642861 29804 raft_consensus.cc:2229] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:17:59.643036 29804 raft_consensus.cc:2258] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:17:59.644160 29804 ts_tablet_manager.cc:1905] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:17:59.645879 29804 ts_tablet_manager.cc:1918] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:17:59.645967 29804 log.cc:1196] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/62e1b6f92abc470588d49b8ecd4907e8
I20260925 05:17:59.646247 29804 ts_tablet_manager.cc:1939] T 62e1b6f92abc470588d49b8ecd4907e8 P e86157afa28545e2a932c03d3dd6cb68: Deleting consensus metadata
W20260925 05:17:59.646294 29289 catalog_manager.cc:4584] TS 337603697ab8457fbdc4998795542193 (127.13.160.129:33465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62e1b6f92abc470588d49b8ecd4907e8: Network error: Client connection negotiation failed: client connection to 127.13.160.129:33465: connect: Connection refused (error 111)
I20260925 05:17:59.646752 29288 catalog_manager.cc:4841] TS e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): tablet 62e1b6f92abc470588d49b8ecd4907e8 (table test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]) successfully deleted
I20260925 05:17:59.647567 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29633
I20260925 05:17:59.653208 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29271
I20260925 05:18:19.659161 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--webserver_interface=127.13.160.190
--webserver_port=42533
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:19.740682 29807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:19.740906 29807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:19.740931 29807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:19.742197 29807 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 05:18:19.742244 29807 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 05:18:19.742259 29807 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 05:18:19.742270 29807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:18:19.742285 29807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:19.742321 29807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:19.742353 29807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:18:19.742369 29807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:18:19.743974 29807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:19.744251 29807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:19.744486 29807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:19.753192 29816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:19.753196 29813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:19.753192 29814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:19.753226 29807 server_base.cc:1034] running on GCE node
I20260925 05:18:19.753844 29807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:19.754191 29807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:19.755332 29807 hybrid_clock.cc:648] HybridClock initialized: now 1790313499755320 us; error 30 us; skew 500 ppm
Sep 25 05:18:19 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:19.762567 29807 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:18:19.763087 29807 webserver.cc:460] Webserver started at http://127.13.160.190:42533/ using document root <none> and password file <none>
I20260925 05:18:19.763365 29807 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:19.763445 29807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:19.764915 29807 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:18:19.765595 29823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:19.765770 29807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:18:19.765870 29807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "63e63f05d8df6aa8119b4a68139f6733"
server_key_iv: "ad6361ede5b1372b344b12a412970c27"
server_key_version: "encryptionkey@0"
I20260925 05:18:19.766291 29807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:19.781843 29807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:19.783953 29807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:19.784117 29807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:19.788100 29807 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:43273
I20260925 05:18:19.788134 29875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:43273 every 8 connection(s)
I20260925 05:18:19.788530 29807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:18:19.790030 29876 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:18:19.790689 29876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap starting.
I20260925 05:18:19.794152 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29807
I20260925 05:18:19.794440 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:33465
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=34089
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:18:19.799196 29876 log.cc:824] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:19.801081 29876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:18:19.801393 29876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap complete.
I20260925 05:18:19.803790 29876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:19.804044 29876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Initialized, Role: FOLLOWER
I20260925 05:18:19.804237 29876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:19.804329 29876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:18:19.804386 29876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:18:19.804435 29876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 1 FOLLOWER]: Advancing to term 2
I20260925 05:18:19.805433 29876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:19.805547 29876 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c4a8ec1b9204cb6baa2ce868e8db9e8; no voters:
I20260925 05:18:19.805754 29876 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 05:18:19.805792 29880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 05:18:19.805902 29880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Running, Role: LEADER
I20260925 05:18:19.806032 29876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:18:19.806048 29880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:19.807282 29881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:18:19.807369 29881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:19.807487 29882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4a8ec1b9204cb6baa2ce868e8db9e8. Latest consensus state: current_term: 2 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:18:19.807821 29882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:19.808068 29894 catalog_manager.cc:1259] Loaded cluster ID: 79e3d3b7c5544eddae77762ab95f81db
I20260925 05:18:19.808130 29894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: loading cluster ID for follower catalog manager: success
I20260925 05:18:19.808938 29894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: acquiring CA information for follower catalog manager: success
I20260925 05:18:19.809303 29894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:18:19.809383 29895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:18:19.809780 29895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:18:19.809862 29895 catalog_manager.cc:1259] Loaded cluster ID: 79e3d3b7c5544eddae77762ab95f81db
I20260925 05:18:19.809900 29895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:18:19.810115 29895 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:18:19.810216 29895 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Loaded TSK: 0
I20260925 05:18:19.834331 29895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Generated new TSK 1
I20260925 05:18:19.834448 29895 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:18:19.886143 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:19.886359 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:19.886384 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:19.887657 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:19.887701 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:19.887729 29878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:18:19.889217 29878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:33465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=34089
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:19.889467 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:19.889703 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:19.894735 29903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:19.894774 29904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:19.894948 29878 server_base.cc:1034] running on GCE node
W20260925 05:18:19.894747 29906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:19.895203 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:19.895474 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:19.896649 29878 hybrid_clock.cc:648] HybridClock initialized: now 1790313499896624 us; error 40 us; skew 500 ppm
Sep 25 05:18:19 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:19.899215 29878 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:18:19.899735 29878 webserver.cc:460] Webserver started at http://127.13.160.129:34089/ using document root <none> and password file <none>
I20260925 05:18:19.899968 29878 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:19.900023 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:19.901393 29878 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:19.902081 29913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:19.902226 29878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:18:19.902328 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "337603697ab8457fbdc4998795542193"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "65d8c663dae0adcec445762981d437c3"
server_key_iv: "c17d69e1c98c03b89e72fd22508e1db5"
server_key_version: "encryptionkey@0"
I20260925 05:18:19.902693 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:19.940461 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:19.942778 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:19.942958 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:19.943243 29878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:19.943703 29920 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 05:18:19.945187 29878 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 05:18:19.945250 29878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 05:18:19.945322 29878 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 05:18:19.945991 29878 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 05:18:19.946049 29878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 05:18:19.946127 29920 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Bootstrap starting.
I20260925 05:18:19.953225 29878 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:33465
I20260925 05:18:19.953357 30027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:33465 every 8 connection(s)
I20260925 05:18:19.953725 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:18:19.955474 29920 log.cc:824] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:19.957938 29920 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:18:19.958287 29920 tablet_bootstrap.cc:492] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Bootstrap complete.
I20260925 05:18:19.959532 29920 ts_tablet_manager.cc:1397] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260925 05:18:19.960143 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 29878
I20260925 05:18:19.960417 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:42113
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=35561
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:18:19.961104 29920 raft_consensus.cc:348] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:18:19.961472 29920 raft_consensus.cc:729] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337603697ab8457fbdc4998795542193, State: Initialized, Role: FOLLOWER
I20260925 05:18:19.961617 29920 consensus_queue.cc:260] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } } peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } }
I20260925 05:18:19.961905 29920 ts_tablet_manager.cc:1428] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Sep 25 05:18:19 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313499, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:19.966464 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:19.954505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:60199)
0925 05:18:19.954719 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:19.954772 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:19.955496 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:19.956045 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:19.956049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:19.956303 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:19.956496 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:19.956501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.958146 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.958147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:19.959628 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:19.959631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.960216 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.960217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:19.960348 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:19.960892 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:19.960900 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:19.962597 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:19.964889 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:19.964896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:19.964898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:19.965128 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:19.965414 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:19.965415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:19.965415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:19.965506 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:19.965993 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:19.965997 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:19.966111 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 05:18:19.966191 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 05:18:19.966312 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260925 05:18:19.966683 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:19.954634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:60199 (local address 127.13.160.190:43273)
0925 05:18:19.954975 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:19.954988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:19.955017 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:18:19.955798 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:19.955803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:19.955848 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:19.955997 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:19.956634 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.957996 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.959706 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.960100 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.960233 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:19.962696 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:19.962700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:19.962700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:19.962712 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:19.964799 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:19.965208 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:19.965213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:19.965214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:19.965345 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:19.965570 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:19.965571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:19.965571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:19.965920 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:19.966021 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:19.966283 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 05:18:19.966444 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":75,"threads_started":1}
I20260925 05:18:19.967358 30028 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:18:19.967473 30028 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:19.967665 30028 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:18:19.968395 29840 ts_manager.cc:194] Registered new tserver with Master: 337603697ab8457fbdc4998795542193 (127.13.160.129:33465)
I20260925 05:18:19.969530 29840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:60199
I20260925 05:18:19.970364 30028 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
I20260925 05:18:19.974129 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:19.969240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.13.160.129:33465)
0925 05:18:19.970981 (+ 1741us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:19.970985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:19.970998 (+ 13us) server_negotiation.cc:407] Connection header received
0925 05:18:19.971024 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:19.971026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:19.971040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:19.971104 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:19.971288 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.971906 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.973363 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.973795 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.973853 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:19.973893 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:19.974053 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 05:18:19.974081 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1713,"thread_start_us":74,"threads_started":1}
I20260925 05:18:19.974169 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:19.968968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.0.0.1:44684)
0925 05:18:19.969171 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:19.969215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:19.969329 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:19.971133 (+ 1804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:19.971137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:19.971140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:19.971215 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:19.971218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.971979 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.971980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:19.973287 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:19.973289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:19.973867 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:19.973868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:19.973949 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:19.973962 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 05:18:19.974034 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 05:18:19.974094 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260925 05:18:19.974562 29962 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e1b6f92abc470588d49b8ecd4907e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:17:59 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:44684
I20260925 05:18:19.975100 30037 tablet_replica.cc:330] stopping tablet replica
I20260925 05:18:19.975209 30037 raft_consensus.cc:2229] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:19.975276 30037 raft_consensus.cc:2258] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:19.975584 30037 ts_tablet_manager.cc:1905] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:19.977787 30037 ts_tablet_manager.cc:1918] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:19.977845 30037 log.cc:1196] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal/wals/62e1b6f92abc470588d49b8ecd4907e8
I20260925 05:18:19.978087 30037 ts_tablet_manager.cc:1939] T 62e1b6f92abc470588d49b8ecd4907e8 P 337603697ab8457fbdc4998795542193: Deleting consensus metadata
I20260925 05:18:19.978551 29826 catalog_manager.cc:4841] TS 337603697ab8457fbdc4998795542193 (127.13.160.129:33465): tablet 62e1b6f92abc470588d49b8ecd4907e8 (table test-table [id=f9ea7cd9c6254232a66c3ff0355565a0]) successfully deleted
W20260925 05:18:20.045558 30033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:20.045815 30033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:20.045856 30033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:20.047091 30033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:20.047153 30033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:20.047199 30033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:18:20.048705 30033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:42113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=35561
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:20.048986 30033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:20.049217 30033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:20.054378 30043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:20.054373 30045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:20.054374 30042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:20.054435 30033 server_base.cc:1034] running on GCE node
I20260925 05:18:20.054867 30033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:20.055122 30033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:20.056260 30033 hybrid_clock.cc:648] HybridClock initialized: now 1790313500056250 us; error 27 us; skew 500 ppm
Sep 25 05:18:20 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:20.058734 30033 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:18:20.059235 30033 webserver.cc:460] Webserver started at http://127.13.160.130:35561/ using document root <none> and password file <none>
I20260925 05:18:20.059489 30033 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:20.059572 30033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:20.060947 30033 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:20.061491 30052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.061666 30033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:18:20.061764 30033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "2c2795a5bf424ec9845e89ee5a28263b"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "1b1bf379794fc3d073ba66e43f454dee"
server_key_iv: "f6742c6fd934d69fb7ed21e7123b5197"
server_key_version: "encryptionkey@0"
I20260925 05:18:20.062103 30033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:20.115449 30033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:20.117548 30033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:20.117735 30033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:20.117985 30033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:20.118311 30033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:20.118363 30033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.118397 30033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:20.118424 30033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.124208 30033 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:42113
I20260925 05:18:20.124253 30165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:42113 every 8 connection(s)
I20260925 05:18:20.124598 30033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:18:20.126983 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30033
I20260925 05:18:20.127279 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:42753
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=37853
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 05:18:20 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:20.135245 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.125315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39255 (local address 127.13.160.190:43273)
0925 05:18:20.125389 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.125391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.125518 (+ 127us) server_negotiation.cc:407] Connection header received
0925 05:18:20.126026 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.126027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.126041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.126121 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:20.127208 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.127790 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.129083 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.129450 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.129522 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.132117 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:20.132119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:20.132119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:20.132127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:20.133701 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.134148 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.134149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.134150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.134213 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.134492 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.134493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.134494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.134615 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:20.134691 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.135023 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.135142 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 05:18:20.135458 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.125255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:39255)
0925 05:18:20.125402 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.125440 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.125927 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.126178 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.126181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.126790 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:20.127104 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.127112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.127876 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.127878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.128991 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.128994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.129574 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.129575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.129732 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.130244 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:20.130255 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:20.132004 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:20.133810 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:20.133818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:20.133820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:20.134069 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:20.134283 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:20.134284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:20.134284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:20.134423 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:20.134682 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:20.134689 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:20.134837 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.134999 (+ 162us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.135192 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260925 05:18:20.135953 30166 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:18:20.136067 30166 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:20.136260 30166 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:18:20.136651 29840 ts_manager.cc:194] Registered new tserver with Master: 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113)
I20260925 05:18:20.137229 29840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:39255
W20260925 05:18:20.210111 30169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:20.210316 30169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:20.210340 30169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:20.211432 30169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:20.211467 30169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:20.211493 30169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:18:20.212845 30169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:42753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=37853
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:20.213032 30169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:20.213209 30169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:20.218094 30175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:20.218086 30174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:20.218123 30169 server_base.cc:1034] running on GCE node
W20260925 05:18:20.218065 30177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:20.218436 30169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:20.218688 30169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:20.219837 30169 hybrid_clock.cc:648] HybridClock initialized: now 1790313500219803 us; error 50 us; skew 500 ppm
Sep 25 05:18:20 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:20.222317 30169 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:18:20.222754 30169 webserver.cc:460] Webserver started at http://127.13.160.131:37853/ using document root <none> and password file <none>
I20260925 05:18:20.223016 30169 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:20.223091 30169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:20.224386 30169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:20.224941 30184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.225092 30169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:18:20.225178 30169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "e86157afa28545e2a932c03d3dd6cb68"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "78af29e4de66a08fc4608307b1173926"
server_key_iv: "0ad457fdccc52b779ad8360b9ba4838c"
server_key_version: "encryptionkey@0"
I20260925 05:18:20.225472 30169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:20.234175 30169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:20.236042 30169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:20.236147 30169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:20.236320 30169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:20.236608 30169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:20.236640 30169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.236681 30169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:20.236707 30169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.242180 30169 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42753
I20260925 05:18:20.242252 30297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42753 every 8 connection(s)
I20260925 05:18:20.242561 30169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
Sep 25 05:18:20 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313500, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:20.251187 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.243266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:38099 (local address 127.13.160.190:43273)
0925 05:18:20.243328 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.243331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.243531 (+ 200us) server_negotiation.cc:407] Connection header received
0925 05:18:20.243971 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.243972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.243983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.244037 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:20.244892 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.245367 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.246320 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.246505 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.246567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.248720 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:20.248721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:20.248722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:20.248728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:20.249909 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.250285 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.250286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.250286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.250333 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.250583 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.250584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.250584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.250721 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:20.250776 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.251056 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.251123 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:18:20.251385 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.243270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:38099)
0925 05:18:20.243409 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.243442 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.243893 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.244090 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.244092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.244635 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:20.244819 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.244824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.245441 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.245442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.246244 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.246247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.246584 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.246585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.246680 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.247224 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:20.247232 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:20.248619 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:20.249989 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:20.249995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:20.249997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:20.250216 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:20.250402 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:20.250403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:20.250404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:20.250511 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:20.250778 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:20.250782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:20.250932 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.251028 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.251147 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260925 05:18:20.251982 30298 heartbeater.cc:347] Connected to a master server at 127.13.160.190:43273
I20260925 05:18:20.252072 30298 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:20.252216 30298 heartbeater.cc:511] Master 127.13.160.190:43273 requested a full tablet report, sending...
I20260925 05:18:20.252586 29840 ts_manager.cc:194] Registered new tserver with Master: e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
I20260925 05:18:20.252734 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30169
I20260925 05:18:20.253027 29840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:38099
I20260925 05:18:20.259111 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.253128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.13.160.190:43273)
0925 05:18:20.253170 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.253172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.253344 (+ 172us) server_negotiation.cc:407] Connection header received
0925 05:18:20.253433 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.253434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.253456 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.253532 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:20.254110 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.254612 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.255388 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.255567 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.255624 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.256848 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:20.256849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:20.256850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:20.256855 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:20.258139 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.258427 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.258428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.258428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.258473 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.258611 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.258611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.258611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.258692 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:20.258846 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.258872 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.258977 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:18:20.259835 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:18:20.262164 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.260181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.13.160.190:43273)
0925 05:18:20.260229 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.260231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.260352 (+ 121us) server_negotiation.cc:407] Connection header received
0925 05:18:20.260397 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.260398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.260427 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.260483 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:18:20.260574 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.261000 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.261670 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.261835 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.261882 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.261990 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:18:20.262028 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 05:18:20.262124 (+ 96us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39454: token expired
Metrics: {"server-negotiator.queue_time_us":30}
W20260925 05:18:20.262228 30031 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39454: token expired
W20260925 05:18:20.262323 29767 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.160.190:43273)
I20260925 05:18:20.267494 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.262564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.13.160.190:43273)
0925 05:18:20.262595 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.262597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.262607 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:18:20.262629 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.262629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.262641 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.262684 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:20.263153 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.263566 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.264178 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.264340 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.264384 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.265432 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:20.265433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:20.265434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:20.265439 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:20.266595 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.266879 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.266880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.266880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.266973 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:20.267115 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:20.267115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:20.267116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:20.267230 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:20.267303 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.267384 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.267437 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 05:18:20.268594 29840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:18:20.270222 29840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:18:20.272289 29962 tablet_service.cc:1467] Processing CreateTablet for tablet 34afd3223be64c5d8043facfc99f4bf4 (DEFAULT_TABLE table=test-table [id=08e3a99739fd44de9b7ec5cb69c508bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:20.272491 29962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34afd3223be64c5d8043facfc99f4bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:20.273883 29920 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Bootstrap starting.
I20260925 05:18:20.274235 29920 tablet_bootstrap.cc:654] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:20.274830 29920 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: No bootstrap required, opened a new log
I20260925 05:18:20.274878 29920 ts_tablet_manager.cc:1397] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:20.275005 29920 raft_consensus.cc:348] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.275075 29920 raft_consensus.cc:374] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:20.275118 29920 raft_consensus.cc:729] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337603697ab8457fbdc4998795542193, State: Initialized, Role: FOLLOWER
I20260925 05:18:20.275164 29920 consensus_queue.cc:260] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.275252 29920 ts_tablet_manager.cc:1428] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:20.276288 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.272021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42753 (local address 127.0.0.1:56620)
0925 05:18:20.272167 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.272177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.272225 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.272724 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.272725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.272727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:20.272771 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.272773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.274046 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.274047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.275203 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.275205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.276042 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.276043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.276117 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.276121 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.276179 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.276219 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260925 05:18:20.276319 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.272381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.13.160.131:42753)
0925 05:18:20.272543 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.272549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.272560 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:18:20.272604 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.272606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.272623 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.272715 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:20.272837 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.273973 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.275309 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.275942 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.276040 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.276101 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.276194 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.276238 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260925 05:18:20.276528 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.272021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:42113 (local address 127.0.0.1:41420)
0925 05:18:20.272182 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.272193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.272234 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.272760 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.272761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.272762 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:20.272810 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.272812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.274256 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.274257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.275251 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.275254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.276257 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.276257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.276302 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.276304 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.276369 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.276423 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":19,"thread_start_us":77,"threads_started":1}
I20260925 05:18:20.276998 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.272176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.13.160.130:42113)
0925 05:18:20.272593 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.272597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.272606 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:20.272641 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.272643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.272659 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.272888 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:20.272979 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.274181 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.275350 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.276107 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.276169 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.276220 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.276865 (+ 645us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.276908 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":90,"threads_started":1}
I20260925 05:18:20.277189 30232 tablet_service.cc:1467] Processing CreateTablet for tablet 34afd3223be64c5d8043facfc99f4bf4 (DEFAULT_TABLE table=test-table [id=08e3a99739fd44de9b7ec5cb69c508bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:20.277558 30232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34afd3223be64c5d8043facfc99f4bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:20.277611 30100 tablet_service.cc:1467] Processing CreateTablet for tablet 34afd3223be64c5d8043facfc99f4bf4 (DEFAULT_TABLE table=test-table [id=08e3a99739fd44de9b7ec5cb69c508bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:20.277941 30100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34afd3223be64c5d8043facfc99f4bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:20.280382 30307 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Bootstrap starting.
I20260925 05:18:20.280673 30308 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Bootstrap starting.
I20260925 05:18:20.281243 30307 tablet_bootstrap.cc:654] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:20.281267 30308 tablet_bootstrap.cc:654] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:20.281735 30307 log.cc:824] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:20.281754 30308 log.cc:824] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:20.282393 30307 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: No bootstrap required, opened a new log
I20260925 05:18:20.282405 30308 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: No bootstrap required, opened a new log
I20260925 05:18:20.282447 30307 ts_tablet_manager.cc:1397] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:18:20.282475 30308 ts_tablet_manager.cc:1397] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:18:20.283921 30308 raft_consensus.cc:348] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.283921 30307 raft_consensus.cc:348] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.284087 30307 raft_consensus.cc:374] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:20.284085 30308 raft_consensus.cc:374] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:20.284132 30308 raft_consensus.cc:729] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2795a5bf424ec9845e89ee5a28263b, State: Initialized, Role: FOLLOWER
I20260925 05:18:20.284161 30307 raft_consensus.cc:729] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86157afa28545e2a932c03d3dd6cb68, State: Initialized, Role: FOLLOWER
I20260925 05:18:20.284281 30308 consensus_queue.cc:260] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.284291 30307 consensus_queue.cc:260] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.284595 30307 ts_tablet_manager.cc:1428] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:18:20.284601 30308 ts_tablet_manager.cc:1428] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:18:20.284636 30166 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
I20260925 05:18:20.284704 30298 heartbeater.cc:503] Master 127.13.160.190:43273 was elected leader, sending a full tablet report...
W20260925 05:18:20.375149 30167 tablet.cc:2369] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:18:20.454449 30029 tablet.cc:2369] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:18:20.493136 30299 tablet.cc:2369] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:18:20.521116 30311 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:20.521250 30311 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.521718 30311 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753), 337603697ab8457fbdc4998795542193 (127.13.160.129:33465)
I20260925 05:18:20.524662 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.522014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:45209 (local address 127.13.160.129:33465)
0925 05:18:20.522134 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.522137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.522147 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:18:20.522179 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.522180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.522191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.522271 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:20.522430 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.522986 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.523911 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.524302 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.524388 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.524397 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.524552 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.524596 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260925 05:18:20.524654 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.521925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.13.160.130:45209)
0925 05:18:20.521980 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.521993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.522037 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.522277 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.522278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.522280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:20.522357 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.522359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.523070 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.523072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.523814 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.523816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.524383 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.524384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.524470 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.524473 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.524534 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.524576 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260925 05:18:20.524736 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.522095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:39905 (local address 127.13.160.131:42753)
0925 05:18:20.522153 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.522155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.522269 (+ 114us) server_negotiation.cc:407] Connection header received
0925 05:18:20.522344 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.522345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.522354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.522419 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:20.522574 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.523062 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.523970 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.524351 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.524437 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.524445 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.524606 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.524643 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 05:18:20.524979 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.522022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42753 (local address 127.13.160.130:39905)
0925 05:18:20.522184 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.522200 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:20.522280 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:20.522423 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:20.522424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:20.522425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:20.522498 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.522499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.523144 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.523145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.523878 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:20.523880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.524438 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.524439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:20.524516 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.524518 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:18:20.524594 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 05:18:20.524636 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260925 05:18:20.525002 29982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34afd3223be64c5d8043facfc99f4bf4" candidate_uuid: "2c2795a5bf424ec9845e89ee5a28263b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337603697ab8457fbdc4998795542193" is_pre_election: true
I20260925 05:18:20.525177 29982 raft_consensus.cc:2454] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2795a5bf424ec9845e89ee5a28263b in term 0.
I20260925 05:18:20.525348 30252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34afd3223be64c5d8043facfc99f4bf4" candidate_uuid: "2c2795a5bf424ec9845e89ee5a28263b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86157afa28545e2a932c03d3dd6cb68" is_pre_election: true
I20260925 05:18:20.525405 30055 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2795a5bf424ec9845e89ee5a28263b, 337603697ab8457fbdc4998795542193; no voters:
I20260925 05:18:20.525518 30252 raft_consensus.cc:2454] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2795a5bf424ec9845e89ee5a28263b in term 0.
I20260925 05:18:20.525569 30311 raft_consensus.cc:2781] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:18:20.525637 30311 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:18:20.525665 30311 raft_consensus.cc:3037] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:20.526404 30311 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.526547 30311 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [CANDIDATE]: Term 1 election: Requested vote from peers e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753), 337603697ab8457fbdc4998795542193 (127.13.160.129:33465)
I20260925 05:18:20.526697 30252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34afd3223be64c5d8043facfc99f4bf4" candidate_uuid: "2c2795a5bf424ec9845e89ee5a28263b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86157afa28545e2a932c03d3dd6cb68"
I20260925 05:18:20.526718 29982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34afd3223be64c5d8043facfc99f4bf4" candidate_uuid: "2c2795a5bf424ec9845e89ee5a28263b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337603697ab8457fbdc4998795542193"
I20260925 05:18:20.526793 30252 raft_consensus.cc:3037] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:20.526790 29982 raft_consensus.cc:3037] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:20.527509 29982 raft_consensus.cc:2454] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2795a5bf424ec9845e89ee5a28263b in term 1.
I20260925 05:18:20.527539 30252 raft_consensus.cc:2454] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2795a5bf424ec9845e89ee5a28263b in term 1.
I20260925 05:18:20.527700 30055 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2795a5bf424ec9845e89ee5a28263b, 337603697ab8457fbdc4998795542193; no voters:
I20260925 05:18:20.527813 30311 raft_consensus.cc:2781] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:18:20.527961 30311 raft_consensus.cc:686] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 LEADER]: Becoming Leader. State: Replica: 2c2795a5bf424ec9845e89ee5a28263b, State: Running, Role: LEADER
I20260925 05:18:20.528081 30311 consensus_queue.cc:237] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:20.528862 29840 catalog_manager.cc:5495] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130). New cstate: current_term: 1 leader_uuid: "2c2795a5bf424ec9845e89ee5a28263b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } health_report { overall_health: UNKNOWN } } }
I20260925 05:18:20.558840 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:20.556569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.13.160.130:42113)
0925 05:18:20.556633 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:20.556635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:20.556645 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:18:20.556688 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:20.556689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:20.556707 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:20.556782 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:18:20.556891 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.557369 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.558086 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:20.558277 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:20.558358 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:20.558496 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:18:20.558597 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:18:20.558690 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:20.558708 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 05:18:20.558727 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:18:20.560731 30252 raft_consensus.cc:1261] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Refusing update from remote peer 2c2795a5bf424ec9845e89ee5a28263b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:20.560731 29982 raft_consensus.cc:1261] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Refusing update from remote peer 2c2795a5bf424ec9845e89ee5a28263b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:20.561100 30315 consensus_queue.cc:1035] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:20.561232 30311 consensus_queue.cc:1035] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [LEADER]: Connected to new peer: Peer: permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:20.564406 30321 mvcc.cc:205] Tried to move back new op lower bound from 7333124098293878784 to 7333124098163507200. Current Snapshot: MvccSnapshot[applied={T|T < 7333124098293878784}]
I20260925 05:18:20.565408 29840 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:20.565505 29840 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:20.566205 29840 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Sending DeleteTablet for 3 replicas of tablet 34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:20.566475 30100 tablet_service.cc:1514] Processing DeleteTablet for tablet 34afd3223be64c5d8043facfc99f4bf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:20 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:41420
I20260925 05:18:20.566542 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29878
I20260925 05:18:20.566629 30232 tablet_service.cc:1514] Processing DeleteTablet for tablet 34afd3223be64c5d8043facfc99f4bf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:20 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:56620
I20260925 05:18:20.567369 30328 tablet_replica.cc:330] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: stopping tablet replica
I20260925 05:18:20.567466 30328 raft_consensus.cc:2229] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:18:20.567574 30328 raft_consensus.cc:2258] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:20.567677 30329 tablet_replica.cc:330] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: stopping tablet replica
I20260925 05:18:20.567761 30329 raft_consensus.cc:2229] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:20.567855 30329 raft_consensus.cc:2258] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:20.568044 30328 ts_tablet_manager.cc:1905] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:20.568287 30329 ts_tablet_manager.cc:1905] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:20.570147 30329 ts_tablet_manager.cc:1918] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:20.570160 30328 ts_tablet_manager.cc:1918] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:20.570230 30329 log.cc:1196] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:20.570261 30328 log.cc:1196] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:20.570554 30328 ts_tablet_manager.cc:1939] T 34afd3223be64c5d8043facfc99f4bf4 P 2c2795a5bf424ec9845e89ee5a28263b: Deleting consensus metadata
I20260925 05:18:20.570555 30329 ts_tablet_manager.cc:1939] T 34afd3223be64c5d8043facfc99f4bf4 P e86157afa28545e2a932c03d3dd6cb68: Deleting consensus metadata
I20260925 05:18:20.571008 29825 catalog_manager.cc:4841] TS e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): tablet 34afd3223be64c5d8043facfc99f4bf4 (table test-table [id=08e3a99739fd44de9b7ec5cb69c508bf]) successfully deleted
I20260925 05:18:20.571115 29824 catalog_manager.cc:4841] TS 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): tablet 34afd3223be64c5d8043facfc99f4bf4 (table test-table [id=08e3a99739fd44de9b7ec5cb69c508bf]) successfully deleted
W20260925 05:18:20.574548 29826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:33465 (error 108)
W20260925 05:18:20.575093 30035 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:18:20.574979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:33465 (local address 127.0.0.1:44686)
0925 05:18:20.575016 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:20.575057 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:33465: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 05:18:20.575340 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30033
W20260925 05:18:20.575253 29826 catalog_manager.cc:4584] TS 337603697ab8457fbdc4998795542193 (127.13.160.129:33465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34afd3223be64c5d8043facfc99f4bf4: Network error: Client connection negotiation failed: client connection to 127.13.160.129:33465: connect: Connection reset by peer (error 104)
I20260925 05:18:20.581419 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30169
I20260925 05:18:20.586316 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 29807
I20260925 05:18:40.591955 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--webserver_interface=127.13.160.190
--webserver_port=42533
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:40.674351 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:40.674599 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:40.674624 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:40.675905 30331 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 05:18:40.675954 30331 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 05:18:40.675967 30331 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 05:18:40.675979 30331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:18:40.676033 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:40.676066 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:40.676083 30331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:18:40.676107 30331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:18:40.677736 30331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:40.677997 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:40.678246 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:40.683595 30339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:40.683761 30331 server_base.cc:1034] running on GCE node
W20260925 05:18:40.683595 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:40.683724 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:40.684167 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:40.684485 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:40.685657 30331 hybrid_clock.cc:648] HybridClock initialized: now 1790313520685656 us; error 43 us; skew 500 ppm
Sep 25 05:18:40 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313520, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:40.692782 30331 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:18:40.693301 30331 webserver.cc:460] Webserver started at http://127.13.160.190:42533/ using document root <none> and password file <none>
I20260925 05:18:40.693547 30331 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:40.693621 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:40.695103 30331 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:40.695837 30346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:40.696013 30331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:40.696125 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "63e63f05d8df6aa8119b4a68139f6733"
server_key_iv: "ad6361ede5b1372b344b12a412970c27"
server_key_version: "encryptionkey@0"
I20260925 05:18:40.696504 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:40.716383 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:40.718359 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:40.718537 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:40.722705 30331 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:43273
I20260925 05:18:40.722748 30398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:43273 every 8 connection(s)
I20260925 05:18:40.723167 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:18:40.724658 30399 sys_catalog.cc:263] Verifying existing consensus state
I20260925 05:18:40.725260 30399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap starting.
I20260925 05:18:40.727393 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30331
I20260925 05:18:40.727730 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:33465
--local_ip_for_outbound_sockets=127.13.160.129
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=34089
--webserver_interface=127.13.160.129
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:18:40.734259 30399 log.cc:824] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:40.736706 30399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:18:40.737056 30399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Bootstrap complete.
I20260925 05:18:40.739476 30399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:40.739710 30399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Initialized, Role: FOLLOWER
I20260925 05:18:40.739890 30399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:40.739954 30399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:18:40.740020 30399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:18:40.740113 30399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 2 FOLLOWER]: Advancing to term 3
I20260925 05:18:40.741178 30399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:40.741295 30399 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c4a8ec1b9204cb6baa2ce868e8db9e8; no voters:
I20260925 05:18:40.741480 30399 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 05:18:40.741518 30403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 05:18:40.741653 30403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [term 3 LEADER]: Becoming Leader. State: Replica: 2c4a8ec1b9204cb6baa2ce868e8db9e8, State: Running, Role: LEADER
I20260925 05:18:40.741740 30403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } }
I20260925 05:18:40.741772 30399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:18:40.742389 30404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4a8ec1b9204cb6baa2ce868e8db9e8. Latest consensus state: current_term: 3 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:18:40.742513 30404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:40.742388 30405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4a8ec1b9204cb6baa2ce868e8db9e8" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 43273 } } }
I20260925 05:18:40.742652 30405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:40.742790 30412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:18:40.743618 30412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:18:40.743722 30412 catalog_manager.cc:1259] Loaded cluster ID: 79e3d3b7c5544eddae77762ab95f81db
I20260925 05:18:40.743745 30412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:18:40.744866 30412 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:18:40.745054 30412 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Loaded TSK: 0
I20260925 05:18:40.745105 30412 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2c4a8ec1b9204cb6baa2ce868e8db9e8: Loaded TSK: 1
F20260925 05:18:40.745328 30412 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1e2c68c9ad google::LogMessage::Fail() at ??:0
@ 0x7f1e2c68f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1e2c68c4de google::LogMessage::Flush() at ??:0
@ 0x7f1e2c68da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1e2edf9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1e2edf6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1e345e61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1e345e648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1e345e6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1e345acc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1e345b6ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1e2e55a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1e2e54f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1e2f3b36db start_thread at ??:0
@ 0x7f1e2b18a61f clone at ??:0
W20260925 05:18:40.844245 30401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:40.844544 30401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:40.844583 30401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:40.846393 30401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:40.846467 30401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:40.846513 30401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:18:40.848893 30401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:33465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=34089
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:40.849195 30401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:40.849483 30401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:40.855044 30440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:40.855046 30441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:40.855037 30443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:40.855837 30401 server_base.cc:1034] running on GCE node
I20260925 05:18:40.856087 30401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:40.856374 30401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:40.857547 30401 hybrid_clock.cc:648] HybridClock initialized: now 1790313520857531 us; error 28 us; skew 500 ppm
Sep 25 05:18:40 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313520, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:40.860744 30401 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:18:40.861326 30401 webserver.cc:460] Webserver started at http://127.13.160.129:34089/ using document root <none> and password file <none>
I20260925 05:18:40.861629 30401 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:40.861698 30401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:40.863564 30401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 05:18:40.864384 30450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:40.864578 30401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:40.864691 30401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "337603697ab8457fbdc4998795542193"
format_stamp: "Formatted at 2026-09-25 05:17:58 on dist-test-slave-9tgz"
server_key: "65d8c663dae0adcec445762981d437c3"
server_key_iv: "c17d69e1c98c03b89e72fd22508e1db5"
server_key_version: "encryptionkey@0"
I20260925 05:18:40.865062 30401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:40.881316 30401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:40.883976 30401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:40.884140 30401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:40.884414 30401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:40.885040 30457 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 05:18:40.887156 30401 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 05:18:40.887239 30401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 05:18:40.887293 30401 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 05:18:40.887998 30401 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 05:18:40.888060 30401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 05:18:40.888150 30457 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Bootstrap starting.
I20260925 05:18:40.895409 30401 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:33465
I20260925 05:18:40.895547 30564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:33465 every 8 connection(s)
I20260925 05:18:40.895874 30401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:18:40.899411 30457 log.cc:824] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:40.902283 30457 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 05:18:40.902628 30457 tablet_bootstrap.cc:492] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Bootstrap complete.
I20260925 05:18:40.903640 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30401
I20260925 05:18:40.903921 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:42113
--local_ip_for_outbound_sockets=127.13.160.130
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=35561
--webserver_interface=127.13.160.130
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:18:40.903949 30457 ts_tablet_manager.cc:1397] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260925 05:18:40.905421 30457 raft_consensus.cc:348] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:40.905791 30457 raft_consensus.cc:729] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337603697ab8457fbdc4998795542193, State: Initialized, Role: FOLLOWER
I20260925 05:18:40.905969 30457 consensus_queue.cc:260] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:40.906275 30457 ts_tablet_manager.cc:1428] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
W20260925 05:18:40.977734 30567 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:18:40.896606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:38785)
0925 05:18:40.896789 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:40.896835 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:40.897326 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:40.977513 (+ 80187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
W20260925 05:18:40.978019 30565 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:43273 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:43273: Client connection negotiation failed: client connection to 127.13.160.190:43273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 05:18:40.978389 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:40.978266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:53797)
0925 05:18:40.978321 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:40.978364 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 05:18:40.978811 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:40.978715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:57369)
0925 05:18:40.978765 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:40.978794 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 05:18:40.978965 30565 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 05:18:40.989224 30569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:40.989439 30569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:40.989463 30569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:40.990710 30569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:40.990751 30569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:40.990808 30569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:18:40.992379 30569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:42113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=35561
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:40.992646 30569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:40.992884 30569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:40.997885 30576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:40.997879 30578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:40.997879 30575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:40.998090 30569 server_base.cc:1034] running on GCE node
I20260925 05:18:40.998392 30569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:40.998672 30569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:40.999840 30569 hybrid_clock.cc:648] HybridClock initialized: now 1790313520999829 us; error 26 us; skew 500 ppm
Sep 25 05:18:40 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313520, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:41.002388 30569 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:18:41.002823 30569 webserver.cc:460] Webserver started at http://127.13.160.130:35561/ using document root <none> and password file <none>
I20260925 05:18:41.003095 30569 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:41.003175 30569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:41.004503 30569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:41.005086 30585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.005270 30569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:18:41.005376 30569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "2c2795a5bf424ec9845e89ee5a28263b"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "1b1bf379794fc3d073ba66e43f454dee"
server_key_iv: "f6742c6fd934d69fb7ed21e7123b5197"
server_key_version: "encryptionkey@0"
I20260925 05:18:41.005757 30569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:41.023237 30569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:41.025727 30569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:41.025905 30569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:41.026157 30569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:41.026474 30569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:41.026530 30569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.026571 30569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:41.026613 30569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.032531 30569 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:42113
I20260925 05:18:41.032651 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:42113 every 8 connection(s)
I20260925 05:18:41.032997 30569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:18:41.034088 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.033702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:36543)
0925 05:18:41.033853 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.033925 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260925 05:18:41.034560 30699 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:43273 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:43273: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
I20260925 05:18:41.034837 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.034746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:49599)
0925 05:18:41.034790 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.034818 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 05:18:41.035142 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.035067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:47597)
0925 05:18:41.035105 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.035127 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 05:18:41.035264 30699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 05:18:41.039597 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30569
I20260925 05:18:41.039830 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:42753
--local_ip_for_outbound_sockets=127.13.160.131
--tserver_master_addrs=127.13.160.190:43273
--webserver_port=37853
--webserver_interface=127.13.160.131
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:41.123010 30702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:41.123302 30702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:41.123344 30702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:41.124600 30702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:41.124688 30702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:41.124737 30702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:18:41.126358 30702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:42753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=37853
--tserver_master_addrs=127.13.160.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:41.126669 30702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:41.126955 30702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:41.132303 30707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:41.132298 30708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:41.132303 30710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:41.132599 30702 server_base.cc:1034] running on GCE node
I20260925 05:18:41.132870 30702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:41.133132 30702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:41.134325 30702 hybrid_clock.cc:648] HybridClock initialized: now 1790313521134309 us; error 29 us; skew 500 ppm
Sep 25 05:18:41 dist-test-slave-9tgz krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313521, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:41.137046 30702 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:18:41.137506 30702 webserver.cc:460] Webserver started at http://127.13.160.131:37853/ using document root <none> and password file <none>
I20260925 05:18:41.137745 30702 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:41.137818 30702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:41.139284 30702 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:18:41.139912 30717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.140094 30702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:18:41.140208 30702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "e86157afa28545e2a932c03d3dd6cb68"
format_stamp: "Formatted at 2026-09-25 05:17:59 on dist-test-slave-9tgz"
server_key: "78af29e4de66a08fc4608307b1173926"
server_key_iv: "0ad457fdccc52b779ad8360b9ba4838c"
server_key_version: "encryptionkey@0"
I20260925 05:18:41.140590 30702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:41.158656 30702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:41.160830 30702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:41.161016 30702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:41.161281 30702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:41.161626 30702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:41.161681 30702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.161722 30702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:41.161765 30702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:41.167834 30702 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42753
I20260925 05:18:41.167894 30830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42753 every 8 connection(s)
I20260925 05:18:41.168311 30702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:18:41.169483 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.169084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:37153)
0925 05:18:41.169236 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.169311 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260925 05:18:41.169919 30831 heartbeater.cc:644] Failed to heartbeat to 127.13.160.190:43273 (0 consecutive failures): Network error: Failed to ping master at 127.13.160.190:43273: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
I20260925 05:18:41.170217 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.170125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:48125)
0925 05:18:41.170172 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.170199 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 05:18:41.170478 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.170405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:59109)
0925 05:18:41.170444 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.170465 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 05:18:41.170593 30831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 05:18:41.175387 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30702
I20260925 05:18:41.979653 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:41.979385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:36351)
0925 05:18:41.979531 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:41.979575 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260925 05:18:42.035820 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.035604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:38813)
0925 05:18:42.035725 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:42.035769 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260925 05:18:42.171258 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.170987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:43625)
0925 05:18:42.171163 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:42.171206 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260925 05:18:42.772884 30838 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:42.773006 30838 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:42.773442 30838 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113), e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
Sep 25 05:18:42 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313520, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.130@KRBTEST.COM
Sep 25 05:18:42 dist-test-slave-9tgz krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313520, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.131@KRBTEST.COM
I20260925 05:18:42.784578 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.773802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:35315 (local address 127.13.160.131:42753)
0925 05:18:42.773962 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:42.773971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:42.774000 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:18:42.774410 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:42.774412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:42.774461 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:42.774618 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:42.775460 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.776300 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777348 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.777756 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777883 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:42.780654 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:42.780660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:42.780660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:42.780673 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:42.782597 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783234 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:42.783237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:42.783238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:42.783341 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783553 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:42.783554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:42.783554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:42.783817 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:42.783893 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:42.784311 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 05:18:42.784454 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":52,"threads_started":1}
I20260925 05:18:42.784598 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.773704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:45089 (local address 127.13.160.130:42113)
0925 05:18:42.773814 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:42.773827 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:42.773862 (+ 35us) server_negotiation.cc:407] Connection header received
0925 05:18:42.774411 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:42.774414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:42.774461 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:42.774618 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:42.775444 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.776300 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777338 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.777756 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777885 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:42.780384 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:42.780389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:42.780389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:42.780410 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:42.782593 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783066 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:42.783069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:42.783069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:42.783159 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783457 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:42.783458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:42.783458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:42.783804 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:42.783893 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:42.784311 (+ 418us) server_negotiation.cc:299] Negotiation successful
0925 05:18:42.784454 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":41,"threads_started":1}
I20260925 05:18:42.784596 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.773660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42753 (local address 127.13.160.129:35315)
0925 05:18:42.773833 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:42.773843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:42.773904 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:42.774663 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:42.774665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:42.775104 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:42.775312 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:42.775319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.776419 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.776420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:42.777241 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:42.777243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.777851 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:42.777981 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:42.778587 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:42.778596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:42.780539 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:42.782964 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:42.782965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:42.782965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:42.783156 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:42.783407 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:42.783408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:42.783472 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:42.783885 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:42.783894 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:42.784086 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 05:18:42.784212 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 05:18:42.784494 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":170,"thread_start_us":50,"threads_started":1}
I20260925 05:18:42.784695 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.773591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:42113 (local address 127.13.160.129:45089)
0925 05:18:42.773789 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:42.773802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:42.773866 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:42.774663 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:42.774665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:42.775104 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:42.775312 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:42.775319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.776415 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.776417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:42.777242 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:42.777245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:42.777853 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:42.777854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:42.777973 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:42.778643 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:42.778648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:42.780255 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:42.782682 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:42.782689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:42.782691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:42.782980 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:42.783255 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:42.783255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:42.783255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:42.783359 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:42.783885 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:42.783894 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:42.784086 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 05:18:42.784212 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 05:18:42.784438 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":99,"thread_start_us":106,"threads_started":1}
W20260925 05:18:42.785557 30451 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
W20260925 05:18:42.785660 30452 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:42.785763 30452 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337603697ab8457fbdc4998795542193; no voters: 2c2795a5bf424ec9845e89ee5a28263b, e86157afa28545e2a932c03d3dd6cb68
I20260925 05:18:42.785877 30838 raft_consensus.cc:2726] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:18:42.980414 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:42.980242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:48461)
0925 05:18:42.980326 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:42.980371 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 05:18:43.036566 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:43.036324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:51985)
0925 05:18:43.036464 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:43.036515 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260925 05:18:43.172024 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:43.171780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:51579)
0925 05:18:43.171921 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:43.171973 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260925 05:18:43.981081 30845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:43.980873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:46589)
0925 05:18:43.980998 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:43.981033 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260925 05:18:44.037397 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:44.037174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:43437)
0925 05:18:44.037304 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:44.037349 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260925 05:18:44.172684 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:44.172442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:52603)
0925 05:18:44.172580 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:44.172625 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260925 05:18:44.589812 30848 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:44.589915 30848 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:44.590106 30848 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113), e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
W20260925 05:18:44.590561 30451 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
W20260925 05:18:44.590667 30452 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:44.590744 30452 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337603697ab8457fbdc4998795542193; no voters: 2c2795a5bf424ec9845e89ee5a28263b, e86157afa28545e2a932c03d3dd6cb68
I20260925 05:18:44.590840 30848 raft_consensus.cc:2726] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:18:44.981926 30849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:44.981627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:60241)
0925 05:18:44.981804 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:44.981857 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260925 05:18:45.038244 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:45.037983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:56095)
0925 05:18:45.038125 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:45.038180 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260925 05:18:45.173445 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:45.173219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:49367)
0925 05:18:45.173351 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:45.173397 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260925 05:18:45.982757 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:45.982504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:37795)
0925 05:18:45.982661 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:45.982709 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260925 05:18:46.039062 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:46.038781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:49235)
0925 05:18:46.038971 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:46.039012 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":63,"threads_started":1}
I20260925 05:18:46.174252 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:46.174016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:33789)
0925 05:18:46.174158 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:46.174206 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260925 05:18:46.972677 30855 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:46.972782 30855 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:46.972949 30855 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113), e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
W20260925 05:18:46.973352 30451 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
W20260925 05:18:46.973433 30452 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:46.973502 30452 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337603697ab8457fbdc4998795542193; no voters: 2c2795a5bf424ec9845e89ee5a28263b, e86157afa28545e2a932c03d3dd6cb68
I20260925 05:18:46.973565 30855 raft_consensus.cc:2726] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:18:46.983453 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:46.983231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:51827)
0925 05:18:46.983373 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:46.983405 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260925 05:18:47.039788 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:47.039577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:52733)
0925 05:18:47.039701 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:47.039742 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260925 05:18:47.175034 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:47.174763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:52679)
0925 05:18:47.174914 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:47.174952 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260925 05:18:47.984176 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:47.983944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:37225)
0925 05:18:47.984077 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:47.984127 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260925 05:18:48.040513 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:48.040286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:33033)
0925 05:18:48.040419 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:48.040464 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260925 05:18:48.175836 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:48.175557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:40071)
0925 05:18:48.175740 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:48.175787 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260925 05:18:48.984931 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:48.984652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:50791)
0925 05:18:48.984789 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:48.984847 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260925 05:18:49.041249 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:49.041019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:42345)
0925 05:18:49.041163 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:49.041200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260925 05:18:49.176651 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:49.176394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:39179)
0925 05:18:49.176551 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:49.176601 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260925 05:18:49.985999 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:49.985701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:55191)
0925 05:18:49.985887 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:49.985934 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260925 05:18:50.042069 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:50.041828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:39853)
0925 05:18:50.041973 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:50.042020 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260925 05:18:50.177397 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:50.177177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:60115)
0925 05:18:50.177302 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:50.177347 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260925 05:18:50.845675 30868 raft_consensus.cc:482] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:50.845821 30868 raft_consensus.cc:504] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2795a5bf424ec9845e89ee5a28263b" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 42113 } } peers { permanent_uuid: "e86157afa28545e2a932c03d3dd6cb68" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42753 } } peers { permanent_uuid: "337603697ab8457fbdc4998795542193" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 33465 } }
I20260925 05:18:50.845997 30868 leader_election.cc:288] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113), e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753)
W20260925 05:18:50.846397 30451 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2c2795a5bf424ec9845e89ee5a28263b (127.13.160.130:42113): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
W20260925 05:18:50.846510 30452 leader_election.cc:341] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e86157afa28545e2a932c03d3dd6cb68 (127.13.160.131:42753): Not found: Tablet not found: 34afd3223be64c5d8043facfc99f4bf4
I20260925 05:18:50.846590 30452 leader_election.cc:302] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 337603697ab8457fbdc4998795542193; no voters: 2c2795a5bf424ec9845e89ee5a28263b, e86157afa28545e2a932c03d3dd6cb68
I20260925 05:18:50.846704 30868 raft_consensus.cc:2726] T 34afd3223be64c5d8043facfc99f4bf4 P 337603697ab8457fbdc4998795542193 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 05:18:50.986981 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:50.986635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:42731)
0925 05:18:50.986810 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:50.986863 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260925 05:18:51.043015 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:51.042711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:32879)
0925 05:18:51.042875 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:51.042966 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260925 05:18:51.178215 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:51.177945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:47759)
0925 05:18:51.178104 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:51.178152 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
I20260925 05:18:51.987699 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:51.987491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:35307)
0925 05:18:51.987615 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:51.987652 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260925 05:18:52.043824 30873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:52.043554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:39489)
0925 05:18:52.043715 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:52.043767 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":63,"threads_started":1}
I20260925 05:18:52.178994 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:52.178727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:39539)
0925 05:18:52.178856 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:52.178938 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260925 05:18:52.988490 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:52.988238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:59945)
0925 05:18:52.988382 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:52.988438 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260925 05:18:53.044634 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:53.044404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:49089)
0925 05:18:53.044545 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:53.044581 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260925 05:18:53.179764 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:53.179512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:52343)
0925 05:18:53.179666 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:53.179713 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260925 05:18:53.989255 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:53.989014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:33423)
0925 05:18:53.989153 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:53.989197 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260925 05:18:54.045419 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:54.045197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:33719)
0925 05:18:54.045325 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:54.045370 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260925 05:18:54.180526 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:54.180313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:40263)
0925 05:18:54.180437 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:54.180478 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260925 05:18:54.990074 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:54.989805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:53269)
0925 05:18:54.989968 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:54.990024 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260925 05:18:55.046249 30882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:55.045994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:56495)
0925 05:18:55.046130 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:55.046182 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260925 05:18:55.181331 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:55.181082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:57775)
0925 05:18:55.181226 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:55.181279 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260925 05:18:55.990844 30884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:55.990641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.129:46007)
0925 05:18:55.990761 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:55.990798 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260925 05:18:56.047035 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:56.046777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.130:54701)
0925 05:18:56.046944 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:56.046989 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260925 05:18:56.181368 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30401
I20260925 05:18:56.182821 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:56.182146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:43273 (local address 127.13.160.131:37389)
0925 05:18:56.182741 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:56.182780 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.190:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":50,"threads_started":1}
I20260925 05:18:56.186471 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30569
I20260925 05:18:56.190780 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30702
2026-09-25T05:18:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58819 ms)
[----------] 30 tests from SecurityITest (235285 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:18:56 dist-test-slave-9tgz krb5kdc[30889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:18:56 dist-test-slave-9tgz krb5kdc[30889](info): set up 2 sockets
Sep 25 05:18:56 dist-test-slave-9tgz krb5kdc[30889](info): commencing operation
krb5kdc: starting...
W20260925 05:18:57.215811 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:18:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:18:57.342739 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33901
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:57.429735 30902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:57.430022 30902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:57.430063 30902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:57.431422 30902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:18:57.431502 30902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:57.431524 30902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:57.431545 30902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:18:57.431577 30902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:18:57.433279 30902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:57.433615 30902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:57.433873 30902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:57.439222 30910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:57.439229 30907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.439352 30902 server_base.cc:1034] running on GCE node
W20260925 05:18:57.439222 30908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.439879 30902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:57.440224 30902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:57.441382 30902 hybrid_clock.cc:648] HybridClock initialized: now 1790313537441360 us; error 40 us; skew 500 ppm
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:57.444620 30902 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:18:57.445122 30902 webserver.cc:460] Webserver started at http://127.13.160.190:41621/ using document root <none> and password file <none>
I20260925 05:18:57.445367 30902 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:57.445416 30902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:57.445569 30902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:57.446738 30902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "473aacb837e4b638345b7fea5749b5fd"
server_key_iv: "a89e3ba8f15d578629fa79c501a8312d"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.447167 30902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "473aacb837e4b638345b7fea5749b5fd"
server_key_iv: "a89e3ba8f15d578629fa79c501a8312d"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.448688 30902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:57.449484 30917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.449676 30902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:57.449782 30902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "473aacb837e4b638345b7fea5749b5fd"
server_key_iv: "a89e3ba8f15d578629fa79c501a8312d"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.449869 30902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:57.465631 30902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:57.467782 30902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:57.467962 30902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:57.472084 30902 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33901
I20260925 05:18:57.472165 30969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33901 every 8 connection(s)
I20260925 05:18:57.472501 30902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:18:57.473436 30970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:57.475943 30970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: Bootstrap starting.
I20260925 05:18:57.476549 30970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:57.476856 30970 log.cc:824] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:57.477602 30970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: No bootstrap required, opened a new log
I20260925 05:18:57.478142 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30902
I20260925 05:18:57.478223 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:18:57.478437 13954 external_mini_cluster.cc:1428] Setting key 6d1086921dce9c121e7155c07d639fd7
I20260925 05:18:57.479652 30970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } }
I20260925 05:18:57.479841 30970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:57.479882 30970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8c49f56ebc454ab7fbeb5cde3208aa, State: Initialized, Role: FOLLOWER
I20260925 05:18:57.480027 30970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } }
I20260925 05:18:57.480118 30970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:18:57.480154 30970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:18:57.480204 30970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:57.481243 30970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } }
I20260925 05:18:57.481381 30970 leader_election.cc:302] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e8c49f56ebc454ab7fbeb5cde3208aa; no voters:
I20260925 05:18:57.481575 30970 leader_election.cc:288] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:18:57.481598 30975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:18:57.481709 30975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [term 1 LEADER]: Becoming Leader. State: Replica: 6e8c49f56ebc454ab7fbeb5cde3208aa, State: Running, Role: LEADER
I20260925 05:18:57.481866 30970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:18:57.481853 30975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } }
I20260925 05:18:57.482287 30975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e8c49f56ebc454ab7fbeb5cde3208aa. Latest consensus state: current_term: 1 leader_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } } }
I20260925 05:18:57.482300 30976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8c49f56ebc454ab7fbeb5cde3208aa" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33901 } } }
I20260925 05:18:57.482483 30975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:57.482496 30976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:57.482733 30984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:18:57.483388 30984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:57.486112 30984 catalog_manager.cc:1347] Generated new cluster ID: 4966278f3cc0440bbe3766688158ee90
I20260925 05:18:57.486177 30984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:18:57.490240 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.479048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.13.160.190:33901)
0925 05:18:57.479232 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:57.479240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:57.479267 (+ 27us) server_negotiation.cc:407] Connection header received
0925 05:18:57.479620 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:57.479627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:57.479683 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:57.479804 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:57.480601 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.481634 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.482733 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.483379 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.483485 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.486301 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:57.486305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:57.486306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:57.486318 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:57.488447 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.488887 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.488891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.488891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.489038 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.489243 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.489244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.489244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.489595 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:57.489762 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:57.489840 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 05:18:57.489995 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260925 05:18:57.499547 30984 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:18:57.500197 30984 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:18:57.504029 30984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: Generated new TSK 0
I20260925 05:18:57.504263 30984 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:18:57.535629 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:57.620810 30996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:57.621040 30996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:57.621076 30996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:57.622354 30996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:57.622409 30996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:57.622438 30996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:18:57.624058 30996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:57.624356 30996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:57.624629 30996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:18:57.630196 30996 server_base.cc:1034] running on GCE node
W20260925 05:18:57.630162 31001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:57.630321 31004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:57.630345 31002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.630733 30996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:57.631083 30996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:57.632277 30996 hybrid_clock.cc:648] HybridClock initialized: now 1790313537632261 us; error 30 us; skew 500 ppm
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:57.635216 30996 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:18:57.635761 30996 webserver.cc:460] Webserver started at http://127.13.160.129:41771/ using document root <none> and password file <none>
I20260925 05:18:57.636009 30996 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:57.636082 30996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:57.636245 30996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:57.637387 30996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "8a5a91657cac4cd8809fc366350f510f"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "2cfbe1b477157c9361b1039b9c954705"
server_key_iv: "a37a4dd7aeb72155abea1667041fd744"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.637780 30996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "8a5a91657cac4cd8809fc366350f510f"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "2cfbe1b477157c9361b1039b9c954705"
server_key_iv: "a37a4dd7aeb72155abea1667041fd744"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.639288 30996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:18:57.640123 31011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.640298 30996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:57.640406 30996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "8a5a91657cac4cd8809fc366350f510f"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "2cfbe1b477157c9361b1039b9c954705"
server_key_iv: "a37a4dd7aeb72155abea1667041fd744"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.640507 30996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:57.653393 30996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:57.656152 30996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:57.656337 30996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:57.656598 30996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:57.656937 30996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:57.656993 30996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.657027 30996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:57.657056 30996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.663422 30996 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:40679
I20260925 05:18:57.663466 31124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:40679 every 8 connection(s)
I20260925 05:18:57.663825 30996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:57.671131 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 30996
I20260925 05:18:57.671262 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:18:57.671468 13954 external_mini_cluster.cc:1428] Setting key 06d1cb9e5d3f56b94b9b29b1b6bf6d2f
I20260925 05:18:57.674353 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.664616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33151 (local address 127.13.160.190:33901)
0925 05:18:57.664681 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:57.664683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:57.664906 (+ 223us) server_negotiation.cc:407] Connection header received
0925 05:18:57.665414 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:57.665415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:57.665429 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:57.665495 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:57.666563 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.667164 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.668185 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.668389 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.668445 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.670714 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:57.670715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:57.670716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:57.670723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:57.672582 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.673161 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.673162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.673163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.673216 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.673517 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.673518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.673518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.673649 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:57.673718 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:57.674106 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 05:18:57.674227 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:18:57.674391 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.664567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33901 (local address 127.13.160.129:33151)
0925 05:18:57.664756 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:57.664802 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:57.665322 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:57.665587 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:57.665591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:57.666243 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:57.666456 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.666461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.667255 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.667255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.668096 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.668099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.668473 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.668474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.668592 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.669143 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:57.669152 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:57.670604 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:57.672671 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.672681 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.672684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.673068 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.673281 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.673282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.673282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.673453 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.673724 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:57.673732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:57.673935 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 05:18:57.674065 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 05:18:57.674208 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260925 05:18:57.675021 31125 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33901
I20260925 05:18:57.675136 31125 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:57.675361 31125 heartbeater.cc:511] Master 127.13.160.190:33901 requested a full tablet report, sending...
I20260925 05:18:57.675897 30934 ts_manager.cc:194] Registered new tserver with Master: 8a5a91657cac4cd8809fc366350f510f (127.13.160.129:40679)
I20260925 05:18:57.676622 30934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:33151
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:18:57.698268 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:57.781623 31130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:57.781932 31130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:57.782012 31130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:57.783456 31130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:57.783511 31130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:57.783541 31130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:18:57.785157 31130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:57.785441 31130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:57.785708 31130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:57.791091 31138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:57.791072 31135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.791317 31130 server_base.cc:1034] running on GCE node
W20260925 05:18:57.791076 31136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.791604 31130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:57.791854 31130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:57.793035 31130 hybrid_clock.cc:648] HybridClock initialized: now 1790313537793010 us; error 36 us; skew 500 ppm
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:57.795593 31130 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:18:57.796126 31130 webserver.cc:460] Webserver started at http://127.13.160.130:40395/ using document root <none> and password file <none>
I20260925 05:18:57.796363 31130 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:57.796432 31130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:57.796588 31130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:57.797704 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "9473d4e4fad3457a8362db5b4f917201"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "7d0e8105ff2bd01ae9ceff05be54df14"
server_key_iv: "6db6a6096291a54e4ebc76f17ed5572e"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.798079 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "9473d4e4fad3457a8362db5b4f917201"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "7d0e8105ff2bd01ae9ceff05be54df14"
server_key_iv: "6db6a6096291a54e4ebc76f17ed5572e"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.799571 31130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:18:57.800403 31145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.800580 31130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:57.800690 31130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "9473d4e4fad3457a8362db5b4f917201"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "7d0e8105ff2bd01ae9ceff05be54df14"
server_key_iv: "6db6a6096291a54e4ebc76f17ed5572e"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.800770 31130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:57.812849 31130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:57.814999 31130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:57.815177 31130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:57.815440 31130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:57.815779 31130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:57.815833 31130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.815867 31130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:57.815898 31130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.821913 31130 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:46681
I20260925 05:18:57.821964 31258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:46681 every 8 connection(s)
I20260925 05:18:57.822309 31130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:18:57.823419 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31130
I20260925 05:18:57.823559 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:18:57.823772 13954 external_mini_cluster.cc:1428] Setting key 5724ab2fd501fa30c3e4d52f947ef53e
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
I20260925 05:18:57.833241 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.823159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:38853 (local address 127.13.160.190:33901)
0925 05:18:57.823242 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:57.823244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:57.823473 (+ 229us) server_negotiation.cc:407] Connection header received
0925 05:18:57.824215 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:57.824216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:57.824226 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:57.824278 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:57.825370 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.825886 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.827126 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.827443 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.827518 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.829815 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:57.829817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:57.829818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:57.829828 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:57.831698 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.832157 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.832158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.832159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.832229 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.832486 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.832488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.832488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.832660 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:57.832744 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:57.833004 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 05:18:57.833100 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:18:57.833326 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.823165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33901 (local address 127.13.160.130:38853)
0925 05:18:57.823338 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:57.823382 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:57.824081 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:57.824358 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:57.824362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:57.825040 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:57.825250 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.825256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.825987 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.825988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.827036 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.827039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.827530 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.827531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.827654 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.828229 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:57.828238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:57.829709 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:57.831804 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.831813 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.831816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.832065 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.832305 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.832306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.832306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.832417 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.832734 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:57.832739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:57.832869 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 05:18:57.832955 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 05:18:57.833090 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260925 05:18:57.833947 31259 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33901
I20260925 05:18:57.834067 31259 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:57.834227 31259 heartbeater.cc:511] Master 127.13.160.190:33901 requested a full tablet report, sending...
I20260925 05:18:57.834604 30934 ts_manager.cc:194] Registered new tserver with Master: 9473d4e4fad3457a8362db5b4f917201 (127.13.160.130:46681)
I20260925 05:18:57.835021 30934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:38853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:18:57.851672 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:38171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:18:57.935863 31264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:57.936123 31264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:57.936151 31264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:57.937331 31264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:57.937377 31264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:57.937407 31264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:18:57.938884 31264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:38171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:57.939124 31264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:57.939319 31264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:57.944756 31270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:57.944738 31269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.944825 31264 server_base.cc:1034] running on GCE node
W20260925 05:18:57.944734 31272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:57.945173 31264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:57.945514 31264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:57.946698 31264 hybrid_clock.cc:648] HybridClock initialized: now 1790313537946686 us; error 44 us; skew 500 ppm
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:18:57.949400 31264 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:18:57.949936 31264 webserver.cc:460] Webserver started at http://127.13.160.131:40785/ using document root <none> and password file <none>
I20260925 05:18:57.950193 31264 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:57.950269 31264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:57.950434 31264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:57.951634 31264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "ab13c4df983c47b88f2de09cc213777c"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "b8cb5e8926babc8acf3cabcaf2a9357f"
server_key_iv: "342f0e6c3c6271d39d7b0a2e6228c2c5"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.952024 31264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "ab13c4df983c47b88f2de09cc213777c"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "b8cb5e8926babc8acf3cabcaf2a9357f"
server_key_iv: "342f0e6c3c6271d39d7b0a2e6228c2c5"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.953620 31264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:18:57.954474 31279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.954662 31264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:57.954768 31264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "ab13c4df983c47b88f2de09cc213777c"
format_stamp: "Formatted at 2026-09-25 05:18:57 on dist-test-slave-9tgz"
server_key: "b8cb5e8926babc8acf3cabcaf2a9357f"
server_key_iv: "342f0e6c3c6271d39d7b0a2e6228c2c5"
server_key_version: "encryptionkey@0"
I20260925 05:18:57.954860 31264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:57.965987 31264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:57.968104 31264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:57.968281 31264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:57.968549 31264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:57.968922 31264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:57.968986 31264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.969029 31264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:57.969055 31264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:57.975116 31264 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:42163
I20260925 05:18:57.975159 31392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:42163 every 8 connection(s)
I20260925 05:18:57.975515 31264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:18:57.977016 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31264
I20260925 05:18:57.977135 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:18:57.977380 13954 external_mini_cluster.cc:1428] Setting key 92e174a30c9096a0e51681e0d8831f55
Sep 25 05:18:57 dist-test-slave-9tgz krb5kdc[30889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313537, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:18:57.985587 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.976317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:47905 (local address 127.13.160.190:33901)
0925 05:18:57.976400 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:57.976403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:57.976588 (+ 185us) server_negotiation.cc:407] Connection header received
0925 05:18:57.977306 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:57.977307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:57.977319 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:57.977385 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:57.978666 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.979248 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.980229 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.980412 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.980464 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.982987 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:57.982988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:57.982989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:57.982995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:57.984360 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.984764 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.984765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.984766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.984816 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.985037 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.985038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.985038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.985142 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:57.985202 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:57.985454 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 05:18:57.985519 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 05:18:57.985796 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.976271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:33901 (local address 127.13.160.131:47905)
0925 05:18:57.976456 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:57.976496 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:57.977154 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:57.977468 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:57.977472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:57.978365 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:57.978577 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.978583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.979327 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.979328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.980148 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:57.980150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.980484 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.980485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:57.980591 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.981171 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:18:57.981181 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:57.982865 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:57.984439 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.984445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.984448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.984696 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.984873 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:18:57.984874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:18:57.984874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:18:57.984979 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:18:57.985206 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:57.985210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:57.985340 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 05:18:57.985427 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 05:18:57.985554 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260925 05:18:57.986276 31393 heartbeater.cc:347] Connected to a master server at 127.13.160.190:33901
I20260925 05:18:57.986366 31393 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:57.986517 31393 heartbeater.cc:511] Master 127.13.160.190:33901 requested a full tablet report, sending...
I20260925 05:18:57.986835 30934 ts_manager.cc:194] Registered new tserver with Master: ab13c4df983c47b88f2de09cc213777c (127.13.160.131:42163)
I20260925 05:18:57.987255 30934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:47905
I20260925 05:18:57.988947 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:18:57.995955 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:57.990360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.13.160.190:33901)
0925 05:18:57.990431 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:57.990433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:57.990444 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:18:57.990506 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:57.990507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:57.990516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:57.990561 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:57.991140 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.991605 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.992275 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:57.992449 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:57.992497 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:57.993709 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:57.993710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:57.993711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:18:57.993715 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:57.994967 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.995276 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.995277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.995277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.995328 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:18:57.995491 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:18:57.995491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:18:57.995491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:18:57.995633 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:57.995710 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:57.995799 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 05:18:57.995860 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:18:57.997666 30934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:18:57.999423 30934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:18:58.006135 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.001987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58652 (local address 127.13.160.129:40679)
0925 05:18:58.002254 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.002259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.002270 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:18:58.002310 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.002313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.002330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.002415 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.002599 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.003866 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005170 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.005677 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005778 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.005841 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.005986 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.006030 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260925 05:18:58.006152 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.001780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40679 (local address 127.0.0.1:58652)
0925 05:18:58.001900 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.001944 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.002017 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.002422 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.002425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.002428 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.002510 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.002512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.003972 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.003973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.005054 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.005055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.005779 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.005867 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.005884 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.005960 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.006019 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":22,"thread_start_us":70,"threads_started":1}
I20260925 05:18:58.006157 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.001998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:42163 (local address 127.0.0.1:43098)
0925 05:18:58.002188 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.002198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.002241 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.002554 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.002555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.002557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.002615 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.002616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.003671 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.003672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.005048 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.005051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.005814 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.005882 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.005886 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.005976 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.006022 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":19,"thread_start_us":42,"threads_started":1}
I20260925 05:18:58.006632 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.002196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.13.160.131:42163)
0925 05:18:58.002382 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.002386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.002395 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:58.002431 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.002433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.002446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.002546 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.002689 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.003560 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005143 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.005682 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005794 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.005854 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.006471 (+ 617us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.006523 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260925 05:18:58.007009 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.001784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:46681 (local address 127.0.0.1:56936)
0925 05:18:58.002050 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.002061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.002113 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.002614 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.002615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.002617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.002671 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.002673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.004110 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.004111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.005139 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.005142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.006778 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.006779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.006837 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.006841 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.006872 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.006941 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260925 05:18:58.007212 31059 tablet_service.cc:1467] Processing CreateTablet for tablet d5f72a041c0c49ea99ead50b628c598a (DEFAULT_TABLE table=test-table [id=8bb98e6dd0374425b112f18db57c5269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.007365 31327 tablet_service.cc:1467] Processing CreateTablet for tablet d5f72a041c0c49ea99ead50b628c598a (DEFAULT_TABLE table=test-table [id=8bb98e6dd0374425b112f18db57c5269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.007555 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.001884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.13.160.130:46681)
0925 05:18:58.002209 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.002215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.002226 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:18:58.002274 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.002276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.002294 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.002799 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.002883 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.003998 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.005892 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.006669 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.007326 (+ 657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.007399 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.007427 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.007474 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":77,"threads_started":1}
I20260925 05:18:58.007602 31059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f72a041c0c49ea99ead50b628c598a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.007715 31327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f72a041c0c49ea99ead50b628c598a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.010423 31407 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Bootstrap starting.
I20260925 05:18:58.010540 31408 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: Bootstrap starting.
I20260925 05:18:58.010731 31193 tablet_service.cc:1467] Processing CreateTablet for tablet d5f72a041c0c49ea99ead50b628c598a (DEFAULT_TABLE table=test-table [id=8bb98e6dd0374425b112f18db57c5269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.011096 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f72a041c0c49ea99ead50b628c598a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.011197 31407 tablet_bootstrap.cc:654] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.011198 31408 tablet_bootstrap.cc:654] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.011794 31408 log.cc:824] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.011816 31407 log.cc:824] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.012622 31408 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: No bootstrap required, opened a new log
I20260925 05:18:58.012683 31408 ts_tablet_manager.cc:1397] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:18:58.012884 31407 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: No bootstrap required, opened a new log
I20260925 05:18:58.012938 31407 ts_tablet_manager.cc:1397] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 05:18:58.013890 31411 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Bootstrap starting.
I20260925 05:18:58.014520 31411 tablet_bootstrap.cc:654] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.014880 31407 raft_consensus.cc:348] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.014880 31408 raft_consensus.cc:348] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.015105 31408 raft_consensus.cc:374] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.015105 31407 raft_consensus.cc:374] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.015143 31407 raft_consensus.cc:729] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab13c4df983c47b88f2de09cc213777c, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.015143 31408 raft_consensus.cc:729] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a5a91657cac4cd8809fc366350f510f, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.015311 31411 log.cc:824] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.015295 31407 consensus_queue.cc:260] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.015305 31408 consensus_queue.cc:260] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.015609 31407 ts_tablet_manager.cc:1428] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:18:58.015609 31408 ts_tablet_manager.cc:1428] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260925 05:18:58.015784 31125 heartbeater.cc:503] Master 127.13.160.190:33901 was elected leader, sending a full tablet report...
I20260925 05:18:58.015799 31393 heartbeater.cc:503] Master 127.13.160.190:33901 was elected leader, sending a full tablet report...
I20260925 05:18:58.016366 31411 tablet_bootstrap.cc:492] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: No bootstrap required, opened a new log
I20260925 05:18:58.016463 31411 ts_tablet_manager.cc:1397] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 05:18:58.018456 31411 raft_consensus.cc:348] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.018648 31411 raft_consensus.cc:374] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.018687 31411 raft_consensus.cc:729] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9473d4e4fad3457a8362db5b4f917201, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.018814 31411 consensus_queue.cc:260] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.019145 31411 ts_tablet_manager.cc:1428] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 05:18:58.019181 31259 heartbeater.cc:503] Master 127.13.160.190:33901 was elected leader, sending a full tablet report...
W20260925 05:18:58.072919 31260 tablet.cc:2369] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:18:58.130035 31413 raft_consensus.cc:482] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:58.130209 31413 raft_consensus.cc:504] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.130630 31413 leader_election.cc:288] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a5a91657cac4cd8809fc366350f510f (127.13.160.129:40679), 9473d4e4fad3457a8362db5b4f917201 (127.13.160.130:46681)
I20260925 05:18:58.133584 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.130982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:50223 (local address 127.13.160.130:46681)
0925 05:18:58.131026 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.131028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.131036 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:18:58.131096 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.131097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.131104 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.131175 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.131341 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.131846 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.132804 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.133205 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.133296 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.133306 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.133465 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.133505 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:18:58.133633 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.130837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:46681 (local address 127.13.160.131:50223)
0925 05:18:58.130910 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.130923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.130975 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.131182 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.131183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.131185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.131262 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.131264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.131944 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.131946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.132697 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.132699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.133295 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.133296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.133383 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.133386 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.133460 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.133505 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 05:18:58.133646 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.130933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:50745 (local address 127.13.160.129:40679)
0925 05:18:58.130985 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.130987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.131151 (+ 164us) server_negotiation.cc:407] Connection header received
0925 05:18:58.131219 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.131220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.131229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.131302 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.131469 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.131931 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.132879 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.133273 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.133362 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.133370 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.133534 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.133579 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 05:18:58.133658 31416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.130839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40679 (local address 127.13.160.131:50745)
0925 05:18:58.131058 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.131068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.131161 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.131306 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.131307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.131308 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.131389 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.131390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.132020 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.132021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.132773 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.132775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.133362 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.133363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.133443 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.133452 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.133516 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.133556 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":21,"thread_start_us":147,"threads_started":1}
I20260925 05:18:58.134037 31079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f72a041c0c49ea99ead50b628c598a" candidate_uuid: "ab13c4df983c47b88f2de09cc213777c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5a91657cac4cd8809fc366350f510f" is_pre_election: true
I20260925 05:18:58.134042 31213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f72a041c0c49ea99ead50b628c598a" candidate_uuid: "ab13c4df983c47b88f2de09cc213777c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9473d4e4fad3457a8362db5b4f917201" is_pre_election: true
I20260925 05:18:58.134212 31213 raft_consensus.cc:2454] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab13c4df983c47b88f2de09cc213777c in term 0.
I20260925 05:18:58.134212 31079 raft_consensus.cc:2454] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab13c4df983c47b88f2de09cc213777c in term 0.
I20260925 05:18:58.134487 31280 leader_election.cc:302] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a5a91657cac4cd8809fc366350f510f, ab13c4df983c47b88f2de09cc213777c; no voters:
I20260925 05:18:58.134641 31413 raft_consensus.cc:2781] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:18:58.134694 31413 raft_consensus.cc:482] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:18:58.134716 31413 raft_consensus.cc:3037] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.135468 31413 raft_consensus.cc:504] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.135632 31413 leader_election.cc:288] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [CANDIDATE]: Term 1 election: Requested vote from peers 8a5a91657cac4cd8809fc366350f510f (127.13.160.129:40679), 9473d4e4fad3457a8362db5b4f917201 (127.13.160.130:46681)
I20260925 05:18:58.135794 31079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f72a041c0c49ea99ead50b628c598a" candidate_uuid: "ab13c4df983c47b88f2de09cc213777c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5a91657cac4cd8809fc366350f510f"
I20260925 05:18:58.135811 31213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f72a041c0c49ea99ead50b628c598a" candidate_uuid: "ab13c4df983c47b88f2de09cc213777c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9473d4e4fad3457a8362db5b4f917201"
I20260925 05:18:58.135890 31213 raft_consensus.cc:3037] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.135890 31079 raft_consensus.cc:3037] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.136871 31213 raft_consensus.cc:2454] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab13c4df983c47b88f2de09cc213777c in term 1.
I20260925 05:18:58.136871 31079 raft_consensus.cc:2454] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab13c4df983c47b88f2de09cc213777c in term 1.
I20260925 05:18:58.137074 31280 leader_election.cc:302] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a5a91657cac4cd8809fc366350f510f, ab13c4df983c47b88f2de09cc213777c; no voters:
I20260925 05:18:58.137215 31413 raft_consensus.cc:2781] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:18:58.137399 31413 raft_consensus.cc:686] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 1 LEADER]: Becoming Leader. State: Replica: ab13c4df983c47b88f2de09cc213777c, State: Running, Role: LEADER
I20260925 05:18:58.137519 31413 consensus_queue.cc:237] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } }
I20260925 05:18:58.138469 30933 catalog_manager.cc:5495] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c reported cstate change: term changed from 0 to 1, leader changed from <none> to ab13c4df983c47b88f2de09cc213777c (127.13.160.131). New cstate: current_term: 1 leader_uuid: "ab13c4df983c47b88f2de09cc213777c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab13c4df983c47b88f2de09cc213777c" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 42163 } health_report { overall_health: HEALTHY } } }
I20260925 05:18:58.153892 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.151360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.13.160.131:42163)
0925 05:18:58.151433 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.151435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.151444 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:58.151493 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.151494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.151513 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.151594 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:18:58.151754 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.152248 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.153042 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.153244 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.153330 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.153514 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:18:58.153629 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:18:58.153731 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.153748 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.153771 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:18:58.155867 31213 raft_consensus.cc:1261] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 1 FOLLOWER]: Refusing update from remote peer ab13c4df983c47b88f2de09cc213777c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:58.155882 31079 raft_consensus.cc:1261] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 1 FOLLOWER]: Refusing update from remote peer ab13c4df983c47b88f2de09cc213777c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:58.156317 31417 consensus_queue.cc:1035] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a5a91657cac4cd8809fc366350f510f" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 40679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:58.156461 31413 consensus_queue.cc:1035] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9473d4e4fad3457a8362db5b4f917201" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:58.159197 31421 mvcc.cc:205] Tried to move back new op lower bound from 7333124252283564032 to 7333124252212137984. Current Snapshot: MvccSnapshot[applied={T|T < 7333124252283564032}]
I20260925 05:18:58.161305 30933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:58.161413 30933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:58.162169 30933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e8c49f56ebc454ab7fbeb5cde3208aa: Sending DeleteTablet for 3 replicas of tablet d5f72a041c0c49ea99ead50b628c598a
I20260925 05:18:58.162557 31059 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f72a041c0c49ea99ead50b628c598a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:58652
I20260925 05:18:58.162587 31193 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f72a041c0c49ea99ead50b628c598a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:56936
I20260925 05:18:58.162802 31327 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f72a041c0c49ea99ead50b628c598a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:43098
I20260925 05:18:58.162848 31429 tablet_replica.cc:330] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f: stopping tablet replica
I20260925 05:18:58.162983 31429 raft_consensus.cc:2229] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:58.163013 31430 tablet_replica.cc:330] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: stopping tablet replica
I20260925 05:18:58.163092 31429 raft_consensus.cc:2258] T d5f72a041c0c49ea99ead50b628c598a P 8a5a91657cac4cd8809fc366350f510f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:58.163111 31430 raft_consensus.cc:2229] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:58.163205 31430 raft_consensus.cc:2258] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:58.163528 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30996
I20260925 05:18:58.164816 31430 ts_tablet_manager.cc:1905] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:58.165086 31431 tablet_replica.cc:330] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: stopping tablet replica
I20260925 05:18:58.165268 31431 raft_consensus.cc:2229] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:18:58.165429 31431 raft_consensus.cc:2258] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:58.165930 31431 ts_tablet_manager.cc:1905] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:58.166934 31430 ts_tablet_manager.cc:1918] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:58.167069 31430 log.cc:1196] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/d5f72a041c0c49ea99ead50b628c598a
I20260925 05:18:58.167433 31430 ts_tablet_manager.cc:1939] T d5f72a041c0c49ea99ead50b628c598a P 9473d4e4fad3457a8362db5b4f917201: Deleting consensus metadata
I20260925 05:18:58.167735 31431 ts_tablet_manager.cc:1918] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:58.167840 31431 log.cc:1196] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/d5f72a041c0c49ea99ead50b628c598a
I20260925 05:18:58.168061 30921 catalog_manager.cc:4841] TS 9473d4e4fad3457a8362db5b4f917201 (127.13.160.130:46681): tablet d5f72a041c0c49ea99ead50b628c598a (table test-table [id=8bb98e6dd0374425b112f18db57c5269]) successfully deleted
I20260925 05:18:58.168146 31431 ts_tablet_manager.cc:1939] T d5f72a041c0c49ea99ead50b628c598a P ab13c4df983c47b88f2de09cc213777c: Deleting consensus metadata
I20260925 05:18:58.168514 30921 catalog_manager.cc:4841] TS ab13c4df983c47b88f2de09cc213777c (127.13.160.131:42163): tablet d5f72a041c0c49ea99ead50b628c598a (table test-table [id=8bb98e6dd0374425b112f18db57c5269]) successfully deleted
W20260925 05:18:58.169895 30918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:40679 (error 108)
I20260925 05:18:58.170363 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.170230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:40679 (local address 127.0.0.1:58664)
0925 05:18:58.170286 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.170328 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 05:18:58.170365 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31130
W20260925 05:18:58.175657 30918 catalog_manager.cc:4584] TS 8a5a91657cac4cd8809fc366350f510f (127.13.160.129:40679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5f72a041c0c49ea99ead50b628c598a: Network error: Client connection negotiation failed: client connection to 127.13.160.129:40679: connect: Connection refused (error 111)
I20260925 05:18:58.176156 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31264
I20260925 05:18:58.181520 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 30902
2026-09-25T05:18:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T05:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:18:58Z Disabled control of system clock
I20260925 05:18:58.209479 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40025
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:40025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:18:58.291906 31436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:58.292171 31436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:58.292194 31436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:58.293329 31436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:18:58.293367 31436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:58.293381 31436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:58.293443 31436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:18:58.293471 31436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:18:58.294831 31436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:40025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:40025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:58.295063 31436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:58.295326 31436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:58.300242 31444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.300227 31442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.300253 31441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.300551 31436 server_base.cc:1034] running on GCE node
I20260925 05:18:58.300800 31436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:58.301069 31436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:58.302213 31436 hybrid_clock.cc:648] HybridClock initialized: now 1790313538302210 us; error 37 us; skew 500 ppm
I20260925 05:18:58.306785 31436 webserver.cc:460] Webserver started at http://127.13.160.190:42925/ using document root <none> and password file <none>
I20260925 05:18:58.307123 31436 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:58.307179 31436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:58.307356 31436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:58.308496 31436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "9076d76984b84537bc7762c46408c5ab"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "4fb38fb207c7b3256919d210b4bb0cd8"
server_key_iv: "a129f5848b2b30f5923953788e6ead16"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.308895 31436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "9076d76984b84537bc7762c46408c5ab"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "4fb38fb207c7b3256919d210b4bb0cd8"
server_key_iv: "a129f5848b2b30f5923953788e6ead16"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.310281 31436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:18:58.311093 31450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.311306 31436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:58.311399 31436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "9076d76984b84537bc7762c46408c5ab"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "4fb38fb207c7b3256919d210b4bb0cd8"
server_key_iv: "a129f5848b2b30f5923953788e6ead16"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.311461 31436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:58.324148 31436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:58.324453 31436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:58.324564 31436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:58.328460 31436 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:40025
I20260925 05:18:58.328497 31502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:40025 every 8 connection(s)
I20260925 05:18:58.328897 31436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:18:58.329902 31503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.332373 31503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: Bootstrap starting.
I20260925 05:18:58.332965 31503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.333264 31503 log.cc:824] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.333988 31503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: No bootstrap required, opened a new log
I20260925 05:18:58.334147 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31436
I20260925 05:18:58.334251 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:18:58.334455 13954 external_mini_cluster.cc:1428] Setting key 6599a5982ded990f4333f83a9e9126f2
I20260925 05:18:58.336040 31503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } }
I20260925 05:18:58.336215 31503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.336280 31503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9076d76984b84537bc7762c46408c5ab, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.336445 31503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } }
I20260925 05:18:58.336544 31503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:18:58.336591 31503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:18:58.336644 31503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.337198 31503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } }
I20260925 05:18:58.337312 31503 leader_election.cc:302] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9076d76984b84537bc7762c46408c5ab; no voters:
I20260925 05:18:58.337489 31503 leader_election.cc:288] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:18:58.337530 31508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:18:58.337693 31508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [term 1 LEADER]: Becoming Leader. State: Replica: 9076d76984b84537bc7762c46408c5ab, State: Running, Role: LEADER
I20260925 05:18:58.337754 31503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:18:58.337818 31508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } }
I20260925 05:18:58.338217 31508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9076d76984b84537bc7762c46408c5ab. Latest consensus state: current_term: 1 leader_uuid: "9076d76984b84537bc7762c46408c5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } } }
I20260925 05:18:58.338449 31508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [sys.catalog]: This master's current role is: LEADER
I20260925 05:18:58.338385 31509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9076d76984b84537bc7762c46408c5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9076d76984b84537bc7762c46408c5ab" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 40025 } } }
I20260925 05:18:58.338828 31509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab [sys.catalog]: This master's current role is: LEADER
W20260925 05:18:58.339439 31524 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:18:58.339510 31524 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:18:58.339592 31517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:18:58.339779 31517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:18:58.341423 31517 catalog_manager.cc:1347] Generated new cluster ID: f7abe0c975af49de9a33511c7cfd5171
I20260925 05:18:58.341439 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.335049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.13.160.190:40025)
0925 05:18:58.335277 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.335284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.335310 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:18:58.336017 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.336024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.336084 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.336223 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:58.336416 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.337932 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.339176 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.339791 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.339918 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.340646 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:58.340650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:58.340651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:18:58.340879 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:58.340923 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:18:58.340937 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:58.341042 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.341141 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.341225 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260925 05:18:58.341485 31517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:18:58.350941 31517 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:18:58.351480 31517 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:18:58.358779 31517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: Generated new TSK 0
I20260925 05:18:58.359005 31517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:18:58.362108 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--builtin_ntp_servers=127.13.160.148:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:18:58.445369 31527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:58.445623 31527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:58.445672 31527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:58.446961 31527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:58.447026 31527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:58.447072 31527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:18:58.448472 31527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:58.448766 31527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:58.449076 31527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:58.454950 31532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.454983 31533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.454957 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.455040 31527 server_base.cc:1034] running on GCE node
I20260925 05:18:58.455415 31527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:58.455720 31527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:58.456887 31527 hybrid_clock.cc:648] HybridClock initialized: now 1790313538456862 us; error 41 us; skew 500 ppm
I20260925 05:18:58.458245 31527 webserver.cc:460] Webserver started at http://127.13.160.129:46555/ using document root <none> and password file <none>
I20260925 05:18:58.458508 31527 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:58.458577 31527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:58.458719 31527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:58.459882 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "615ca88e75bb44d0a46d54a9615d7a12"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "aac843779b99fbfa98e2c4f67e0aa02d"
server_key_iv: "9d70997feba5d54d3093f16d2a14cbcc"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.460268 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "615ca88e75bb44d0a46d54a9615d7a12"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "aac843779b99fbfa98e2c4f67e0aa02d"
server_key_iv: "9d70997feba5d54d3093f16d2a14cbcc"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.461687 31527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:18:58.462484 31541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.462728 31527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:58.462841 31527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "615ca88e75bb44d0a46d54a9615d7a12"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "aac843779b99fbfa98e2c4f67e0aa02d"
server_key_iv: "9d70997feba5d54d3093f16d2a14cbcc"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.462956 31527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:58.483287 31527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:58.483621 31527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:58.483775 31527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:58.484026 31527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:58.484364 31527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:58.484416 31527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.484458 31527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:58.484480 31527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.490334 31527 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:36063
I20260925 05:18:58.490386 31654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:36063 every 8 connection(s)
I20260925 05:18:58.490763 31527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:18:58.496912 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.491580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:36771 (local address 127.13.160.190:40025)
0925 05:18:58.491647 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.491649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.491904 (+ 255us) server_negotiation.cc:407] Connection header received
0925 05:18:58.492460 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.492461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.492475 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.492530 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:58.493327 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.494308 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.495356 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.495580 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.495672 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.496581 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:58.496582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:58.496583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:18:58.496590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:58.496606 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:18:58.496611 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:58.496692 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.496799 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.496830 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:18:58.497123 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.491563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40025 (local address 127.13.160.129:36771)
0925 05:18:58.491772 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.491811 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.492350 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.492610 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.492614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.492624 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:58.493216 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.493222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.494423 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.494424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.495261 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.495264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.495688 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.495689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.495823 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.496422 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:18:58.496423 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:58.496453 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:18:58.496478 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:58.496690 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:58.496697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:58.496700 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.496786 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.496889 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260925 05:18:58.497493 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31527
I20260925 05:18:58.497605 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:18:58.497802 13954 external_mini_cluster.cc:1428] Setting key 80e2695db1b3d1d0b2c8eedc54208a07
I20260925 05:18:58.497956 31655 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40025
I20260925 05:18:58.498063 31655 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:58.498276 31655 heartbeater.cc:511] Master 127.13.160.190:40025 requested a full tablet report, sending...
I20260925 05:18:58.498764 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--builtin_ntp_servers=127.13.160.148:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 05:18:58.498821 31467 ts_manager.cc:194] Registered new tserver with Master: 615ca88e75bb44d0a46d54a9615d7a12 (127.13.160.129:36063)
I20260925 05:18:58.499728 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.129:36771
W20260925 05:18:58.581066 31658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:58.581329 31658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:58.581365 31658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:58.582518 31658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:58.582583 31658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:58.582623 31658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:18:58.584043 31658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:58.584312 31658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:58.584607 31658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:58.589725 31664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.589874 31658 server_base.cc:1034] running on GCE node
W20260925 05:18:58.589847 31663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.589721 31666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.590248 31658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:58.590535 31658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:58.591712 31658 hybrid_clock.cc:648] HybridClock initialized: now 1790313538591703 us; error 39 us; skew 500 ppm
I20260925 05:18:58.593055 31658 webserver.cc:460] Webserver started at http://127.13.160.130:37155/ using document root <none> and password file <none>
I20260925 05:18:58.593331 31658 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:58.593402 31658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:58.593565 31658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:58.594650 31658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "cc05cb7a5d1542eab201770aa9cec414"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "6234cb64c901edab4ff949d75b299d51"
server_key_iv: "1c78db593b571bf3b55526c43571a10b"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.595052 31658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "cc05cb7a5d1542eab201770aa9cec414"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "6234cb64c901edab4ff949d75b299d51"
server_key_iv: "1c78db593b571bf3b55526c43571a10b"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.596360 31658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:18:58.597062 31672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.597232 31658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:18:58.597335 31658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "cc05cb7a5d1542eab201770aa9cec414"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "6234cb64c901edab4ff949d75b299d51"
server_key_iv: "1c78db593b571bf3b55526c43571a10b"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.597414 31658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:58.607051 31658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:58.607385 31658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:58.607543 31658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:58.607790 31658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:58.608131 31658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:58.608187 31658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.608222 31658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:58.608266 31658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.614003 31658 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:35807
I20260925 05:18:58.614079 31785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:35807 every 8 connection(s)
I20260925 05:18:58.614481 31658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:18:58.619591 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.615308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:57817 (local address 127.13.160.190:40025)
0925 05:18:58.615383 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.615385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.615606 (+ 221us) server_negotiation.cc:407] Connection header received
0925 05:18:58.616087 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.616088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.616098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.616168 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:58.616789 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.617280 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.618231 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.618415 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.618469 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.619384 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:58.619384 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:58.619385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:18:58.619389 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:58.619399 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:18:58.619402 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:58.619442 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.619502 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.619527 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:18:58.619835 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.615302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40025 (local address 127.13.160.130:57817)
0925 05:18:58.615476 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.615517 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.616003 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.616226 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.616229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.616237 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:58.616718 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.616723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.617361 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.617362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.618152 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.618154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.618492 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.618493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.618602 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.619257 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:18:58.619258 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:58.619290 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:18:58.619312 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:58.619443 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:58.619445 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:58.619448 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.619496 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.619576 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260925 05:18:58.620355 31786 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40025
I20260925 05:18:58.620455 31786 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:58.620599 31786 heartbeater.cc:511] Master 127.13.160.190:40025 requested a full tablet report, sending...
I20260925 05:18:58.620966 31467 ts_manager.cc:194] Registered new tserver with Master: cc05cb7a5d1542eab201770aa9cec414 (127.13.160.130:35807)
I20260925 05:18:58.621359 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.130:57817
I20260925 05:18:58.623783 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31658
I20260925 05:18:58.623903 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:18:58.624094 13954 external_mini_cluster.cc:1428] Setting key 481ee14ee32bc78165d363fd7103b77b
I20260925 05:18:58.624985 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--builtin_ntp_servers=127.13.160.148:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:18:58.705853 31789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:18:58.706077 31789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:18:58.706100 31789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:18:58.707208 31789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:18:58.707242 31789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:18:58.707266 31789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:18:58.708531 31789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:40025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:18:58.708727 31789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:18:58.708981 31789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:18:58.714038 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:18:58.714084 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.714201 31789 server_base.cc:1034] running on GCE node
W20260925 05:18:58.714051 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:18:58.714488 31789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:18:58.714769 31789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:18:58.715945 31789 hybrid_clock.cc:648] HybridClock initialized: now 1790313538715932 us; error 28 us; skew 500 ppm
I20260925 05:18:58.717247 31789 webserver.cc:460] Webserver started at http://127.13.160.131:36911/ using document root <none> and password file <none>
I20260925 05:18:58.717504 31789 fs_manager.cc:359] Metadata directory not provided
I20260925 05:18:58.717577 31789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:18:58.717734 31789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:18:58.718812 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "882bf51a10664c7fafa87c1fbbbdd268"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "29bf01365cd9407587887dc0a924c0de"
server_key_iv: "85dab7f8abeb733761db1886babdda9a"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.719260 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "882bf51a10664c7fafa87c1fbbbdd268"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "29bf01365cd9407587887dc0a924c0de"
server_key_iv: "85dab7f8abeb733761db1886babdda9a"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.720510 31789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:18:58.721195 31803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.721354 31789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:18:58.721457 31789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "882bf51a10664c7fafa87c1fbbbdd268"
format_stamp: "Formatted at 2026-09-25 05:18:58 on dist-test-slave-9tgz"
server_key: "29bf01365cd9407587887dc0a924c0de"
server_key_iv: "85dab7f8abeb733761db1886babdda9a"
server_key_version: "encryptionkey@0"
I20260925 05:18:58.721539 31789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:18:58.749591 31789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:18:58.749931 31789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:18:58.750037 31789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:18:58.750232 31789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:18:58.750594 31789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:18:58.750653 31789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.750689 31789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:18:58.750738 31789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:18:58.756407 31789 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:36153
I20260925 05:18:58.756466 31916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:36153 every 8 connection(s)
I20260925 05:18:58.756851 31789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:18:58.759986 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31789
I20260925 05:18:58.760098 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:18:58.760298 13954 external_mini_cluster.cc:1428] Setting key 03952b1c76f36a5fada257ea830eeaf4
I20260925 05:18:58.762439 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.757611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:44347 (local address 127.13.160.190:40025)
0925 05:18:58.757695 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.757697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.757941 (+ 244us) server_negotiation.cc:407] Connection header received
0925 05:18:58.758417 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.758417 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.758428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.758479 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:58.759227 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.759719 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.761215 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.761422 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.761475 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.762230 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:58.762231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:58.762231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:18:58.762236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:58.762249 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:18:58.762253 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:58.762301 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.762352 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.762379 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 05:18:58.762647 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.757570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:40025 (local address 127.13.160.131:44347)
0925 05:18:58.757775 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.757834 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.758330 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.758534 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.758537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.758545 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:18:58.759149 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.759156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.759828 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.759830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.761153 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.761156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.761494 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.761495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.761601 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.762099 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:18:58.762099 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:18:58.762128 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:18:58.762150 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:18:58.762303 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:18:58.762305 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:18:58.762308 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.762355 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.762433 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260925 05:18:58.763156 31917 heartbeater.cc:347] Connected to a master server at 127.13.160.190:40025
I20260925 05:18:58.763247 31917 heartbeater.cc:464] Registering TS with master...
I20260925 05:18:58.763429 31917 heartbeater.cc:511] Master 127.13.160.190:40025 requested a full tablet report, sending...
I20260925 05:18:58.763813 31467 ts_manager.cc:194] Registered new tserver with Master: 882bf51a10664c7fafa87c1fbbbdd268 (127.13.160.131:36153)
I20260925 05:18:58.764135 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.131:44347
I20260925 05:18:58.772006 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:18:58.776420 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.773375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.13.160.190:40025)
0925 05:18:58.773465 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.773468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.773556 (+ 88us) server_negotiation.cc:407] Connection header received
0925 05:18:58.773622 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.773624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.773636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.773708 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:18:58.773890 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.774381 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.775247 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.775438 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.775510 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.776145 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:18:58.776146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:18:58.776147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:18:58.776159 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:18:58.776176 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:18:58.776188 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:18:58.776250 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.776309 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.776333 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 05:18:58.778021 31467 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:18:58.779706 31467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:18:58.786075 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.782256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:36063 (local address 127.0.0.1:43462)
0925 05:18:58.782355 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.782364 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.782401 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.782808 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.782808 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.782810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.782856 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.782857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783705 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.783706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.784994 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.784997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.785859 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.785918 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.785923 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.785971 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.785999 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260925 05:18:58.786083 31927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.782067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:36153 (local address 127.0.0.1:57566)
0925 05:18:58.782204 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.782213 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.782259 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.782499 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.782499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.782501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.782530 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.782531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783932 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.783933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.784846 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.784848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.785758 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.785812 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.785845 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.785922 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.785983 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
I20260925 05:18:58.786293 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.782165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.13.160.131:36153)
0925 05:18:58.782342 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.782345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.782354 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:58.782391 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.782393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.782408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.782613 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.782665 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783817 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.784932 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.785664 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785761 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.785861 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.785938 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.786183 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260925 05:18:58.786671 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.782456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.13.160.129:36063)
0925 05:18:58.782661 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.782666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.782676 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:18:58.782715 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.782718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.782733 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.782811 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.782926 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783613 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785088 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.785781 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785852 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.785925 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.786528 (+ 603us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.786570 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260925 05:18:58.787238 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.781842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:35807 (local address 127.0.0.1:53238)
0925 05:18:58.781992 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.782037 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.782114 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.782448 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.782452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.782454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.782526 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.782528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783899 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.783900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.784994 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.784997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.787006 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.787007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.787084 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.787088 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.787119 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.787163 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":31,"thread_start_us":74,"threads_started":1}
I20260925 05:18:58.787101 31851 tablet_service.cc:1467] Processing CreateTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf (DEFAULT_TABLE table=test-table [id=9a13916d6a53472d8c29f75c6d0093af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.787253 31589 tablet_service.cc:1467] Processing CreateTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf (DEFAULT_TABLE table=test-table [id=9a13916d6a53472d8c29f75c6d0093af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.787499 31851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510076e810ef42b1bcaf1b2f6475fbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.787528 31589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510076e810ef42b1bcaf1b2f6475fbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.787662 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.781976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.13.160.130:35807)
0925 05:18:58.782263 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.782268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.782277 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:58.782318 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.782321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.782337 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.782604 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.782664 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.783817 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.785962 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.786885 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.787418 (+ 533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.787517 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.787540 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.787582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260925 05:18:58.788203 31720 tablet_service.cc:1467] Processing CreateTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf (DEFAULT_TABLE table=test-table [id=9a13916d6a53472d8c29f75c6d0093af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:18:58.788501 31720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510076e810ef42b1bcaf1b2f6475fbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:18:58.790984 31931 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Bootstrap starting.
I20260925 05:18:58.791687 31931 tablet_bootstrap.cc:654] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.792016 31931 log.cc:824] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.792160 31933 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Bootstrap starting.
I20260925 05:18:58.792337 31932 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: Bootstrap starting.
I20260925 05:18:58.792780 31933 tablet_bootstrap.cc:654] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.792830 31931 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: No bootstrap required, opened a new log
I20260925 05:18:58.792887 31931 ts_tablet_manager.cc:1397] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:18:58.792995 31932 tablet_bootstrap.cc:654] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: Neither blocks nor log segments found. Creating new log.
I20260925 05:18:58.793128 31933 log.cc:824] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.793309 31932 log.cc:824] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: Log is configured to *not* fsync() on all Append() calls
I20260925 05:18:58.793880 31933 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: No bootstrap required, opened a new log
I20260925 05:18:58.793965 31933 ts_tablet_manager.cc:1397] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:18:58.795449 31932 tablet_bootstrap.cc:492] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: No bootstrap required, opened a new log
I20260925 05:18:58.795516 31932 ts_tablet_manager.cc:1397] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 05:18:58.795395 31931 raft_consensus.cc:348] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.795572 31931 raft_consensus.cc:374] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.795622 31931 raft_consensus.cc:729] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882bf51a10664c7fafa87c1fbbbdd268, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.795739 31931 consensus_queue.cc:260] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.796021 31931 ts_tablet_manager.cc:1428] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:18:58.796186 31917 heartbeater.cc:503] Master 127.13.160.190:40025 was elected leader, sending a full tablet report...
I20260925 05:18:58.796430 31933 raft_consensus.cc:348] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.796592 31933 raft_consensus.cc:374] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.796629 31933 raft_consensus.cc:729] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc05cb7a5d1542eab201770aa9cec414, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.796736 31933 consensus_queue.cc:260] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.797003 31933 ts_tablet_manager.cc:1428] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:18:58.797117 31786 heartbeater.cc:503] Master 127.13.160.190:40025 was elected leader, sending a full tablet report...
I20260925 05:18:58.798144 31932 raft_consensus.cc:348] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.798262 31932 raft_consensus.cc:374] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:18:58.798283 31932 raft_consensus.cc:729] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615ca88e75bb44d0a46d54a9615d7a12, State: Initialized, Role: FOLLOWER
I20260925 05:18:58.798354 31932 consensus_queue.cc:260] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.798532 31932 ts_tablet_manager.cc:1428] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:18:58.798620 31655 heartbeater.cc:503] Master 127.13.160.190:40025 was elected leader, sending a full tablet report...
W20260925 05:18:58.865094 31787 tablet.cc:2369] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:18:58.920881 31939 raft_consensus.cc:482] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:18:58.921010 31939 raft_consensus.cc:504] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.921403 31939 leader_election.cc:288] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc05cb7a5d1542eab201770aa9cec414 (127.13.160.130:35807), 882bf51a10664c7fafa87c1fbbbdd268 (127.13.160.131:36153)
I20260925 05:18:58.924489 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.921655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:35807 (local address 127.13.160.129:33717)
0925 05:18:58.921720 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.921733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.921781 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.921997 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.921998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.922020 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.922106 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.922108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.922811 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.922812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.923613 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.923615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.924260 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.924261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.924337 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.924341 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.924400 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.924435 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 05:18:58.924544 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.921749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:33717 (local address 127.13.160.130:35807)
0925 05:18:58.921807 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.921810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.921821 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:18:58.921857 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.921858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.921871 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.921948 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.922190 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.922728 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.923738 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.924188 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.924244 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.924251 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.924417 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.924458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 05:18:58.924649 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.921785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:36153 (local address 127.13.160.129:42789)
0925 05:18:58.921950 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.921962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:18:58.922020 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:18:58.922191 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:18:58.922192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:18:58.922193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:18:58.922259 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.922261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.922885 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.922886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.923724 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:18:58.923727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.924359 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.924360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:18:58.924463 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.924467 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:18:58.924525 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 05:18:58.924561 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260925 05:18:58.924688 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.921907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:42789 (local address 127.13.160.131:36153)
0925 05:18:58.921956 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.921958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.922034 (+ 76us) server_negotiation.cc:407] Connection header received
0925 05:18:58.922085 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.922085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.922098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.922157 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:18:58.922331 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.922806 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.923812 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.924272 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.924341 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.924350 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.924554 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.924602 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:18:58.924954 31871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510076e810ef42b1bcaf1b2f6475fbdf" candidate_uuid: "615ca88e75bb44d0a46d54a9615d7a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882bf51a10664c7fafa87c1fbbbdd268" is_pre_election: true
I20260925 05:18:58.924961 31740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510076e810ef42b1bcaf1b2f6475fbdf" candidate_uuid: "615ca88e75bb44d0a46d54a9615d7a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc05cb7a5d1542eab201770aa9cec414" is_pre_election: true
I20260925 05:18:58.925112 31740 raft_consensus.cc:2454] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615ca88e75bb44d0a46d54a9615d7a12 in term 0.
I20260925 05:18:58.925112 31871 raft_consensus.cc:2454] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615ca88e75bb44d0a46d54a9615d7a12 in term 0.
I20260925 05:18:58.925346 31543 leader_election.cc:302] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615ca88e75bb44d0a46d54a9615d7a12, cc05cb7a5d1542eab201770aa9cec414; no voters:
I20260925 05:18:58.925477 31939 raft_consensus.cc:2781] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:18:58.925531 31939 raft_consensus.cc:482] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:18:58.925561 31939 raft_consensus.cc:3037] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.926265 31939 raft_consensus.cc:504] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.926397 31939 leader_election.cc:288] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [CANDIDATE]: Term 1 election: Requested vote from peers cc05cb7a5d1542eab201770aa9cec414 (127.13.160.130:35807), 882bf51a10664c7fafa87c1fbbbdd268 (127.13.160.131:36153)
I20260925 05:18:58.926586 31740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510076e810ef42b1bcaf1b2f6475fbdf" candidate_uuid: "615ca88e75bb44d0a46d54a9615d7a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc05cb7a5d1542eab201770aa9cec414"
I20260925 05:18:58.926579 31871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510076e810ef42b1bcaf1b2f6475fbdf" candidate_uuid: "615ca88e75bb44d0a46d54a9615d7a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882bf51a10664c7fafa87c1fbbbdd268"
I20260925 05:18:58.926661 31740 raft_consensus.cc:3037] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.926682 31871 raft_consensus.cc:3037] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:18:58.927490 31871 raft_consensus.cc:2454] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615ca88e75bb44d0a46d54a9615d7a12 in term 1.
I20260925 05:18:58.927500 31740 raft_consensus.cc:2454] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615ca88e75bb44d0a46d54a9615d7a12 in term 1.
I20260925 05:18:58.927659 31543 leader_election.cc:302] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615ca88e75bb44d0a46d54a9615d7a12, cc05cb7a5d1542eab201770aa9cec414; no voters:
I20260925 05:18:58.927764 31939 raft_consensus.cc:2781] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:18:58.927929 31939 raft_consensus.cc:686] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [term 1 LEADER]: Becoming Leader. State: Replica: 615ca88e75bb44d0a46d54a9615d7a12, State: Running, Role: LEADER
I20260925 05:18:58.928027 31939 consensus_queue.cc:237] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } }
I20260925 05:18:58.928939 31466 catalog_manager.cc:5495] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 615ca88e75bb44d0a46d54a9615d7a12 (127.13.160.129). New cstate: current_term: 1 leader_uuid: "615ca88e75bb44d0a46d54a9615d7a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615ca88e75bb44d0a46d54a9615d7a12" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 36063 } health_report { overall_health: HEALTHY } } }
I20260925 05:18:58.968549 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.966258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.13.160.129:36063)
0925 05:18:58.966329 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:18:58.966331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:18:58.966340 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:18:58.966415 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:18:58.966416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:18:58.966437 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:18:58.966491 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:18:58.966626 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.967128 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.967838 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:18:58.968018 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:18:58.968066 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:18:58.968202 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:18:58.968303 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:18:58.968365 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:18:58.968403 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 05:18:58.968421 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:18:58.970429 31871 raft_consensus.cc:1261] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 1 FOLLOWER]: Refusing update from remote peer 615ca88e75bb44d0a46d54a9615d7a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:58.970443 31740 raft_consensus.cc:1261] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 1 FOLLOWER]: Refusing update from remote peer 615ca88e75bb44d0a46d54a9615d7a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:18:58.970795 31939 consensus_queue.cc:1035] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "882bf51a10664c7fafa87c1fbbbdd268" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 36153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:58.970999 31941 consensus_queue.cc:1035] T 510076e810ef42b1bcaf1b2f6475fbdf P 615ca88e75bb44d0a46d54a9615d7a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc05cb7a5d1542eab201770aa9cec414" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:18:58.974067 31945 mvcc.cc:205] Tried to move back new op lower bound from 7333124255619862528 to 7333124255449812992. Current Snapshot: MvccSnapshot[applied={T|T < 7333124255619862528}]
I20260925 05:18:58.974112 31946 mvcc.cc:205] Tried to move back new op lower bound from 7333124255619862528 to 7333124255449812992. Current Snapshot: MvccSnapshot[applied={T|T < 7333124255619862528}]
I20260925 05:18:58.976462 31466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:58.976570 31466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:18:58.977375 31466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9076d76984b84537bc7762c46408c5ab: Sending DeleteTablet for 3 replicas of tablet 510076e810ef42b1bcaf1b2f6475fbdf
I20260925 05:18:58.977743 31720 tablet_service.cc:1514] Processing DeleteTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='slave'} at 127.0.0.1:53238
I20260925 05:18:58.977849 31589 tablet_service.cc:1514] Processing DeleteTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='slave'} at 127.0.0.1:43462
I20260925 05:18:58.977851 31851 tablet_service.cc:1514] Processing DeleteTablet for tablet 510076e810ef42b1bcaf1b2f6475fbdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:18:58 UTC) from {username='slave'} at 127.0.0.1:57566
I20260925 05:18:58.978040 31954 tablet_replica.cc:330] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: stopping tablet replica
I20260925 05:18:58.978101 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31527
I20260925 05:18:58.978215 31954 raft_consensus.cc:2229] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:58.978353 31954 raft_consensus.cc:2258] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:58.978785 31956 tablet_replica.cc:330] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: stopping tablet replica
I20260925 05:18:58.978912 31956 raft_consensus.cc:2229] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:18:58.979024 31956 raft_consensus.cc:2258] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:18:58.979223 31954 ts_tablet_manager.cc:1905] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:58.979478 31956 ts_tablet_manager.cc:1905] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:18:58.981485 31956 ts_tablet_manager.cc:1918] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:58.981485 31954 ts_tablet_manager.cc:1918] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:18:58.981572 31954 log.cc:1196] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/510076e810ef42b1bcaf1b2f6475fbdf
I20260925 05:18:58.981572 31956 log.cc:1196] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/510076e810ef42b1bcaf1b2f6475fbdf
I20260925 05:18:58.981842 31956 ts_tablet_manager.cc:1939] T 510076e810ef42b1bcaf1b2f6475fbdf P 882bf51a10664c7fafa87c1fbbbdd268: Deleting consensus metadata
I20260925 05:18:58.981842 31954 ts_tablet_manager.cc:1939] T 510076e810ef42b1bcaf1b2f6475fbdf P cc05cb7a5d1542eab201770aa9cec414: Deleting consensus metadata
I20260925 05:18:58.982295 31452 catalog_manager.cc:4841] TS 882bf51a10664c7fafa87c1fbbbdd268 (127.13.160.131:36153): tablet 510076e810ef42b1bcaf1b2f6475fbdf (table test-table [id=9a13916d6a53472d8c29f75c6d0093af]) successfully deleted
I20260925 05:18:58.982414 31452 catalog_manager.cc:4841] TS cc05cb7a5d1542eab201770aa9cec414 (127.13.160.130:35807): tablet 510076e810ef42b1bcaf1b2f6475fbdf (table test-table [id=9a13916d6a53472d8c29f75c6d0093af]) successfully deleted
W20260925 05:18:58.984504 31452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:36063 (error 108)
I20260925 05:18:58.985011 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31658
I20260925 05:18:58.985034 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:18:58.984901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:36063 (local address 127.0.0.1:43474)
0925 05:18:58.984958 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 05:18:58.985001 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:36063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 05:18:58.985178 31452 catalog_manager.cc:4584] TS 615ca88e75bb44d0a46d54a9615d7a12 (127.13.160.129:36063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 510076e810ef42b1bcaf1b2f6475fbdf: Network error: Client connection negotiation failed: client connection to 127.13.160.129:36063: connect: Connection refused (error 111)
I20260925 05:18:58.990424 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31789
I20260925 05:18:58.995262 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31436
2026-09-25T05:18:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (812 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:18:59 dist-test-slave-9tgz krb5kdc[31960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:18:59 dist-test-slave-9tgz krb5kdc[31960](info): set up 2 sockets
Sep 25 05:18:59 dist-test-slave-9tgz krb5kdc[31960](info): commencing operation
krb5kdc: starting...
W20260925 05:19:00.028774 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:00.153769 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37843
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:37843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:00.237614 31973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:00.237884 31973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:00.237926 31973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:00.239241 31973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:00.239320 31973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:00.239346 31973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:00.239375 31973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:00.239408 31973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:00.240932 31973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:37843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:00.241230 31973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:00.241482 31973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:00.246424 31979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.246441 31978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.246451 31981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.246829 31973 server_base.cc:1034] running on GCE node
I20260925 05:19:00.247110 31973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:00.247378 31973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:00.248476 31973 hybrid_clock.cc:648] HybridClock initialized: now 1790313540248463 us; error 30 us; skew 500 ppm
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:00.251057 31973 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:00.251504 31973 webserver.cc:460] Webserver started at http://127.13.160.190:33519/ using document root <none> and password file <none>
I20260925 05:19:00.251765 31973 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:00.251812 31973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:00.251986 31973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:00.253131 31973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "748fd23abe754de79d8c3ad4f9126f8a"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "dc8ad9df8ebf49b7cf379003b34ca7df"
server_key_iv: "d75886ad27845e95643d5553b75b07fe"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.253502 31973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "748fd23abe754de79d8c3ad4f9126f8a"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "dc8ad9df8ebf49b7cf379003b34ca7df"
server_key_iv: "d75886ad27845e95643d5553b75b07fe"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.254930 31973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:00.255723 31988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.255900 31973 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:00.256003 31973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "748fd23abe754de79d8c3ad4f9126f8a"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "dc8ad9df8ebf49b7cf379003b34ca7df"
server_key_iv: "d75886ad27845e95643d5553b75b07fe"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.256089 31973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:00.271869 31973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:00.273923 31973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:00.274106 31973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:00.278290 31973 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:37843
I20260925 05:19:00.278329 32040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:37843 every 8 connection(s)
I20260925 05:19:00.278730 31973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:00.279071 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 31973
I20260925 05:19:00.279196 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:00.279428 13954 external_mini_cluster.cc:1428] Setting key f6a0f3f5a495639de51dba2999668df5
I20260925 05:19:00.279945 32041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:00.283336 32041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: Bootstrap starting.
I20260925 05:19:00.284031 32041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:00.284359 32041 log.cc:824] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:00.285117 32041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: No bootstrap required, opened a new log
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:00.287160 32041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } }
I20260925 05:19:00.287352 32041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:00.287393 32041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748fd23abe754de79d8c3ad4f9126f8a, State: Initialized, Role: FOLLOWER
I20260925 05:19:00.287531 32041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } }
I20260925 05:19:00.287634 32041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:00.287717 32041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:00.287775 32041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:00.288502 32041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } }
I20260925 05:19:00.288597 32041 leader_election.cc:302] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 748fd23abe754de79d8c3ad4f9126f8a; no voters:
I20260925 05:19:00.288861 32041 leader_election.cc:288] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:00.288897 32046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:00.289011 32046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [term 1 LEADER]: Becoming Leader. State: Replica: 748fd23abe754de79d8c3ad4f9126f8a, State: Running, Role: LEADER
I20260925 05:19:00.289160 32041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:00.289155 32046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } }
I20260925 05:19:00.289563 32046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 748fd23abe754de79d8c3ad4f9126f8a. Latest consensus state: current_term: 1 leader_uuid: "748fd23abe754de79d8c3ad4f9126f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } } }
I20260925 05:19:00.289635 32046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:00.289719 32047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "748fd23abe754de79d8c3ad4f9126f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748fd23abe754de79d8c3ad4f9126f8a" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37843 } } }
I20260925 05:19:00.289811 32047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:00.290009 32050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:00.290619 32050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:00.291193 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.280020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42054 (local address 127.13.160.190:37843)
0925 05:19:00.280208 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.280218 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.280249 (+ 31us) server_negotiation.cc:407] Connection header received
0925 05:19:00.280801 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.280810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.280878 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.281040 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:00.281642 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.283063 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.283885 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.284529 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.284646 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.287031 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:00.287038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:00.287038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:00.287055 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:00.289058 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.289485 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.289488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.289489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.289595 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.289835 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.289836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.289837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.290162 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:00.290732 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.290810 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.290983 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260925 05:19:00.295625 32050 catalog_manager.cc:1347] Generated new cluster ID: 5ccb9aa3c8b24f329a6e5273cd903dd5
I20260925 05:19:00.295717 32050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:00.307242 32050 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:00.308045 32050 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:00.314821 32050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: Generated new TSK 0
I20260925 05:19:00.315080 32050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:19:00.346040 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:00.429947 32067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:00.430187 32067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:00.430228 32067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:00.431468 32067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:00.431531 32067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:00.431577 32067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:19:00.433036 32067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:00.433297 32067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:00.433527 32067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:00.438601 32073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.438601 32072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.438694 32075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.438709 32067 server_base.cc:1034] running on GCE node
I20260925 05:19:00.439098 32067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:00.439340 32067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:00.440510 32067 hybrid_clock.cc:648] HybridClock initialized: now 1790313540440490 us; error 39 us; skew 500 ppm
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:00.443011 32067 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:19:00.443473 32067 webserver.cc:460] Webserver started at http://127.13.160.129:43565/ using document root <none> and password file <none>
I20260925 05:19:00.443656 32067 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:00.443698 32067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:00.443794 32067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:00.444795 32067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "f2a075f25fef40aba5732a8f6187c278"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "7e68b5af1d615afa444787425ef7462f"
server_key_iv: "b1fa62e589d8c21d0901c92321a67e04"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.445098 32067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "f2a075f25fef40aba5732a8f6187c278"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "7e68b5af1d615afa444787425ef7462f"
server_key_iv: "b1fa62e589d8c21d0901c92321a67e04"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.446339 32067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:00.447033 32082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.447201 32067 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:00.447280 32067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "f2a075f25fef40aba5732a8f6187c278"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "7e68b5af1d615afa444787425ef7462f"
server_key_iv: "b1fa62e589d8c21d0901c92321a67e04"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.447333 32067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:00.504480 32067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:00.506417 32067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:00.506532 32067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:00.506726 32067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:00.507098 32067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:00.507156 32067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.507191 32067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:00.507232 32067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.513087 32067 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:39813
I20260925 05:19:00.513137 32195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:39813 every 8 connection(s)
I20260925 05:19:00.513464 32067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:00.522271 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32067
I20260925 05:19:00.522404 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:19:00.522639 13954 external_mini_cluster.cc:1428] Setting key 54429f85374b70d06e6dad6874dd6c05
I20260925 05:19:00.522852 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.514193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:34205 (local address 127.13.160.190:37843)
0925 05:19:00.514262 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.514264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.514536 (+ 272us) server_negotiation.cc:407] Connection header received
0925 05:19:00.515114 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.515116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.515128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.515185 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:00.516163 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.516688 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.517689 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.517907 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.517995 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.520276 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:00.520278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:00.520278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:00.520285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:00.521470 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.521835 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.521835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.521836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.521883 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.522134 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.522134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.522135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.522246 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:00.522300 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.522644 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.522737 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:19:00.523036 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.514161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:37843 (local address 127.13.160.129:34205)
0925 05:19:00.514348 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.514431 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.515003 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.515260 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.515264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.515859 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:00.516055 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.516059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.516788 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.516789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.517593 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.517595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.518006 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.518007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.518140 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.518689 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:00.518698 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:00.520181 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:00.521541 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.521547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.521549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.521770 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.521945 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.521945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.521945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.522043 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.522313 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:00.522320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:00.522496 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.522624 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.522801 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260925 05:19:00.523553 32196 heartbeater.cc:347] Connected to a master server at 127.13.160.190:37843
I20260925 05:19:00.523649 32196 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:00.523833 32196 heartbeater.cc:511] Master 127.13.160.190:37843 requested a full tablet report, sending...
I20260925 05:19:00.524405 32005 ts_manager.cc:194] Registered new tserver with Master: f2a075f25fef40aba5732a8f6187c278 (127.13.160.129:39813)
I20260925 05:19:00.525262 32005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:34205
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:19:00.549239 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:00.630429 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:00.630659 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:00.630695 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:00.631856 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:00.631927 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:00.631971 32201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:19:00.633394 32201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:00.633667 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:00.633908 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:00.638907 32207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.638863 32209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.638943 32201 server_base.cc:1034] running on GCE node
W20260925 05:19:00.638875 32206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.639369 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:00.639628 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:00.640767 32201 hybrid_clock.cc:648] HybridClock initialized: now 1790313540640760 us; error 33 us; skew 500 ppm
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:00.643257 32201 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:19:00.643675 32201 webserver.cc:460] Webserver started at http://127.13.160.130:33373/ using document root <none> and password file <none>
I20260925 05:19:00.643939 32201 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:00.644003 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:00.644148 32201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:00.645165 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "e932c446244947bda9960388cb7497a7"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "fefdd1f7fb5ff2e51c79f56bb501d549"
server_key_iv: "9b5fe88a1ddae196ae4dd57c6f9f767f"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.645524 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "e932c446244947bda9960388cb7497a7"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "fefdd1f7fb5ff2e51c79f56bb501d549"
server_key_iv: "9b5fe88a1ddae196ae4dd57c6f9f767f"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.646812 32201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:19:00.647564 32216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.647727 32201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:00.647830 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "e932c446244947bda9960388cb7497a7"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "fefdd1f7fb5ff2e51c79f56bb501d549"
server_key_iv: "9b5fe88a1ddae196ae4dd57c6f9f767f"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.647907 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:00.668694 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:00.670765 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:00.670964 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:00.671211 32201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:00.671531 32201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:00.671584 32201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.671623 32201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:00.671653 32201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.677327 32201 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:33911
I20260925 05:19:00.677428 32329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:33911 every 8 connection(s)
I20260925 05:19:00.677731 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:00.684629 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32201
I20260925 05:19:00.684753 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:19:00.684998 13954 external_mini_cluster.cc:1428] Setting key d4d7fbddd175d8cf3653df419f2bff63
I20260925 05:19:00.686993 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.678429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:38845 (local address 127.13.160.190:37843)
0925 05:19:00.678500 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.678502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.678730 (+ 228us) server_negotiation.cc:407] Connection header received
0925 05:19:00.679240 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.679241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.679250 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.679315 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:00.680202 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.680685 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.681618 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.681811 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.681860 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.684087 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:00.684089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:00.684089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:00.684096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:00.685584 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.686002 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.686003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.686003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.686068 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.686336 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.686337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.686337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.686456 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:00.686532 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.686795 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.686878 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:19:00.687203 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.678408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:37843 (local address 127.13.160.130:38845)
0925 05:19:00.678583 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.678624 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.679147 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.679370 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.679373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.679928 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:00.680123 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.680128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.680766 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.680767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.681542 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.681544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.681880 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.681881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.681985 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.682522 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:00.682530 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:00.683996 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:00.685666 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.685673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.685675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.685920 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.686139 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.686140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.686140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.686259 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.686523 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:00.686530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:00.686662 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.686775 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.686951 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260925 05:19:00.687863 32330 heartbeater.cc:347] Connected to a master server at 127.13.160.190:37843
I20260925 05:19:00.687947 32330 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:00.688141 32330 heartbeater.cc:511] Master 127.13.160.190:37843 requested a full tablet report, sending...
I20260925 05:19:00.688516 32005 ts_manager.cc:194] Registered new tserver with Master: e932c446244947bda9960388cb7497a7 (127.13.160.130:33911)
I20260925 05:19:00.688963 32005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:38845
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:19:00.709719 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:00.791929 32335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:00.792138 32335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:00.792161 32335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:00.793342 32335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:00.793385 32335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:00.793438 32335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:19:00.795039 32335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:37843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:00.795281 32335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:00.795511 32335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:00.800485 32340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.800611 32335 server_base.cc:1034] running on GCE node
W20260925 05:19:00.800498 32341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:00.800704 32343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:00.800931 32335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:00.801236 32335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:00.802414 32335 hybrid_clock.cc:648] HybridClock initialized: now 1790313540802399 us; error 44 us; skew 500 ppm
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:00.805025 32335 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:19:00.805509 32335 webserver.cc:460] Webserver started at http://127.13.160.131:44541/ using document root <none> and password file <none>
I20260925 05:19:00.805749 32335 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:00.805817 32335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:00.805970 32335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:00.807111 32335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "55dce3cdb4204cce831c485bb56f14be"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "c6f560ddcbe07925d7cd4f5d63ca6e57"
server_key_iv: "96799c2ac9662ab5af1581f390f4bcee"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.807473 32335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "55dce3cdb4204cce831c485bb56f14be"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "c6f560ddcbe07925d7cd4f5d63ca6e57"
server_key_iv: "96799c2ac9662ab5af1581f390f4bcee"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.808799 32335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:00.809504 32350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.809679 32335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:00.809783 32335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "55dce3cdb4204cce831c485bb56f14be"
format_stamp: "Formatted at 2026-09-25 05:19:00 on dist-test-slave-9tgz"
server_key: "c6f560ddcbe07925d7cd4f5d63ca6e57"
server_key_iv: "96799c2ac9662ab5af1581f390f4bcee"
server_key_version: "encryptionkey@0"
I20260925 05:19:00.809857 32335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:00.830849 32335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:00.832785 32335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:00.832955 32335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:00.833209 32335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:00.833527 32335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:00.833582 32335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.833627 32335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:00.833652 32335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:00.839679 32335 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:45997
I20260925 05:19:00.839773 32463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:45997 every 8 connection(s)
I20260925 05:19:00.840178 32335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:19:00.844913 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32335
I20260925 05:19:00.845012 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:19:00.845194 13954 external_mini_cluster.cc:1428] Setting key ecdf4af7e1ca530ffde7657749e0447d
Sep 25 05:19:00 dist-test-slave-9tgz krb5kdc[31960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313540, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:00.849993 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.841323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:41491 (local address 127.13.160.190:37843)
0925 05:19:00.841390 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.841393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.841601 (+ 208us) server_negotiation.cc:407] Connection header received
0925 05:19:00.842069 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.842070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.842080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.842132 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:00.843150 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.843635 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.844658 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.844844 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.844899 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.847380 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:00.847382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:00.847382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:00.847388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:00.848804 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.849142 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.849143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.849143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.849191 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.849449 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.849450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.849450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.849567 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:00.849619 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.849864 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.849927 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:19:00.850212 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.841308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:37843 (local address 127.13.160.131:41491)
0925 05:19:00.841472 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.841510 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.841986 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.842194 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.842197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.842768 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:00.843048 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.843054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.843719 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.843720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.844576 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.844578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.844934 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.844935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.845096 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.845727 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:00.845749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:00.847300 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:00.848871 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.848876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.848878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.849080 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.849257 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:00.849258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:00.849258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:00.849351 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:00.849625 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:00.849630 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:00.849750 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.849836 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.849955 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260925 05:19:00.850840 32464 heartbeater.cc:347] Connected to a master server at 127.13.160.190:37843
I20260925 05:19:00.850975 32464 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:00.851132 32464 heartbeater.cc:511] Master 127.13.160.190:37843 requested a full tablet report, sending...
I20260925 05:19:00.851507 32005 ts_manager.cc:194] Registered new tserver with Master: 55dce3cdb4204cce831c485bb56f14be (127.13.160.131:45997)
I20260925 05:19:00.851982 32005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:41491
I20260925 05:19:00.857059 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:19:00.864667 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.858436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.13.160.190:37843)
0925 05:19:00.858485 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.858487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.858624 (+ 137us) server_negotiation.cc:407] Connection header received
0925 05:19:00.858681 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.858681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.858694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.858762 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:00.859352 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.859984 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.860688 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.860872 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.860923 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.862308 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:00.862309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:00.862309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:00.862314 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:00.863583 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.863981 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.863982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.863982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.864035 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:00.864219 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:00.864219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:00.864220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:00.864352 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:00.864430 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.864551 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.864600 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:00.866226 32005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:19:00.867910 32005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:19:00.874548 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:45997 (local address 127.0.0.1:44554)
0925 05:19:00.870683 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.870727 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.870803 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.871104 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.871108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.871110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:00.871197 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.871200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872135 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.872136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.873363 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.873365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874230 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.874311 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874323 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.874392 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.874447 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260925 05:19:00.874657 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39813 (local address 127.0.0.1:50518)
0925 05:19:00.870691 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.870725 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.870803 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.871107 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.871112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.871115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:00.871199 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.871201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872493 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.872494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.873544 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.873545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874399 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.874453 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874456 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.874514 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.874548 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":36,"thread_start_us":55,"threads_started":1}
I20260925 05:19:00.874915 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.13.160.131:45997)
0925 05:19:00.870685 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.870691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.870812 (+ 121us) server_negotiation.cc:407] Connection header received
0925 05:19:00.870916 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.870919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.870937 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.871016 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:00.871289 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872036 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.873450 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874125 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874228 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874288 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.874767 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.874812 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260925 05:19:00.875015 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.13.160.130:33911)
0925 05:19:00.871108 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.871113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.871122 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:00.871156 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.871158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.871174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.871247 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:00.871452 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872518 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.873868 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874577 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874645 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874700 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.874865 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.874935 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":86,"threads_started":1}
I20260925 05:19:00.875025 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33911 (local address 127.0.0.1:45756)
0925 05:19:00.870932 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.870941 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.870980 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.871330 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.871331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.871333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:00.871367 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.871368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872631 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.872632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.873775 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.873778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874736 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.874793 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874797 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.874848 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.874924 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":102,"threads_started":1}
I20260925 05:19:00.875619 32398 tablet_service.cc:1467] Processing CreateTablet for tablet 887b58ba86594c548cfa1d202df18308 (DEFAULT_TABLE table=test-table [id=a95262a4340f452885e1180e2dec2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:00.875735 32264 tablet_service.cc:1467] Processing CreateTablet for tablet 887b58ba86594c548cfa1d202df18308 (DEFAULT_TABLE table=test-table [id=a95262a4340f452885e1180e2dec2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:00.875970 32398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 887b58ba86594c548cfa1d202df18308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:00.876113 32264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 887b58ba86594c548cfa1d202df18308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:00.878484 32481 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Bootstrap starting.
I20260925 05:19:00.878692 32482 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Bootstrap starting.
I20260925 05:19:00.878806 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.870490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.13.160.129:39813)
0925 05:19:00.870766 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.870770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.870781 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:00.870932 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.870935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.870952 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.871025 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:00.871263 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.872386 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.873626 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.874315 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.874395 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.874446 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.874530 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.874573 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":80,"threads_started":1}
I20260925 05:19:00.879441 32481 tablet_bootstrap.cc:654] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:00.879505 32130 tablet_service.cc:1467] Processing CreateTablet for tablet 887b58ba86594c548cfa1d202df18308 (DEFAULT_TABLE table=test-table [id=a95262a4340f452885e1180e2dec2d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:00.879877 32130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 887b58ba86594c548cfa1d202df18308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:00.880043 32481 log.cc:824] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:00.880750 32481 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: No bootstrap required, opened a new log
I20260925 05:19:00.880836 32481 ts_tablet_manager.cc:1397] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:00.880955 32482 tablet_bootstrap.cc:654] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:00.881507 32482 log.cc:824] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:00.882261 32482 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: No bootstrap required, opened a new log
I20260925 05:19:00.882333 32482 ts_tablet_manager.cc:1397] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 05:19:00.882654 32481 raft_consensus.cc:348] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.882823 32481 raft_consensus.cc:374] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:00.882866 32481 raft_consensus.cc:729] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e932c446244947bda9960388cb7497a7, State: Initialized, Role: FOLLOWER
I20260925 05:19:00.883006 32481 consensus_queue.cc:260] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.883291 32481 ts_tablet_manager.cc:1428] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 05:19:00.883420 32330 heartbeater.cc:503] Master 127.13.160.190:37843 was elected leader, sending a full tablet report...
I20260925 05:19:00.883373 32486 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: Bootstrap starting.
I20260925 05:19:00.884219 32486 tablet_bootstrap.cc:654] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:00.884362 32482 raft_consensus.cc:348] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.884495 32482 raft_consensus.cc:374] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:00.884516 32482 raft_consensus.cc:729] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55dce3cdb4204cce831c485bb56f14be, State: Initialized, Role: FOLLOWER
I20260925 05:19:00.884639 32482 consensus_queue.cc:260] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.884821 32486 log.cc:824] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:00.884912 32464 heartbeater.cc:503] Master 127.13.160.190:37843 was elected leader, sending a full tablet report...
I20260925 05:19:00.884910 32482 ts_tablet_manager.cc:1428] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:00.885648 32486 tablet_bootstrap.cc:492] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: No bootstrap required, opened a new log
I20260925 05:19:00.885730 32486 ts_tablet_manager.cc:1397] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:00.887169 32486 raft_consensus.cc:348] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.887296 32486 raft_consensus.cc:374] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:00.887317 32486 raft_consensus.cc:729] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a075f25fef40aba5732a8f6187c278, State: Initialized, Role: FOLLOWER
I20260925 05:19:00.887456 32486 consensus_queue.cc:260] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.887732 32486 ts_tablet_manager.cc:1428] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:00.887777 32196 heartbeater.cc:503] Master 127.13.160.190:37843 was elected leader, sending a full tablet report...
I20260925 05:19:00.890141 32487 raft_consensus.cc:482] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:19:00.890216 32487 raft_consensus.cc:504] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.890560 32487 leader_election.cc:288] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2a075f25fef40aba5732a8f6187c278 (127.13.160.129:39813), e932c446244947bda9960388cb7497a7 (127.13.160.130:33911)
I20260925 05:19:00.894372 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.890910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:51649 (local address 127.13.160.130:33911)
0925 05:19:00.890952 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.890954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.891057 (+ 103us) server_negotiation.cc:407] Connection header received
0925 05:19:00.891094 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.891095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.891102 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.891143 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:00.891286 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.891963 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.893436 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.894017 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.894097 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.894106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.894269 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.894305 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:19:00.894372 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.890814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:58769 (local address 127.13.160.129:39813)
0925 05:19:00.890850 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.890853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.890870 (+ 17us) server_negotiation.cc:407] Connection header received
0925 05:19:00.890894 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.890895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.890928 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.890974 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:00.891205 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.891959 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.893435 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.894018 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.894097 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.894106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.894268 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.894304 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:19:00.894382 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.890726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39813 (local address 127.13.160.131:58769)
0925 05:19:00.890783 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.890795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.890843 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.891025 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.891026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.891028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:00.891117 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.891119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.892043 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.892044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.893343 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.893346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.894106 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.894107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.894189 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.894191 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.894242 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.894287 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":45}
I20260925 05:19:00.894804 32150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "887b58ba86594c548cfa1d202df18308" candidate_uuid: "55dce3cdb4204cce831c485bb56f14be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a075f25fef40aba5732a8f6187c278" is_pre_election: true
I20260925 05:19:00.894392 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.890827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:33911 (local address 127.13.160.131:51649)
0925 05:19:00.890987 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.890997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:00.891045 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:00.891155 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:00.891156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:00.891157 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:00.891222 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.891223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.892043 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.892044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.893344 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:00.893346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.894106 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.894107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:00.894186 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.894190 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:00.894253 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 05:19:00.894286 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":80,"thread_start_us":120,"threads_started":1}
I20260925 05:19:00.895022 32150 raft_consensus.cc:2454] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55dce3cdb4204cce831c485bb56f14be in term 0.
I20260925 05:19:00.895228 32284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "887b58ba86594c548cfa1d202df18308" candidate_uuid: "55dce3cdb4204cce831c485bb56f14be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e932c446244947bda9960388cb7497a7" is_pre_election: true
I20260925 05:19:00.895285 32351 leader_election.cc:302] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55dce3cdb4204cce831c485bb56f14be, f2a075f25fef40aba5732a8f6187c278; no voters:
I20260925 05:19:00.895361 32284 raft_consensus.cc:2454] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55dce3cdb4204cce831c485bb56f14be in term 0.
I20260925 05:19:00.895408 32487 raft_consensus.cc:2781] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:19:00.895499 32487 raft_consensus.cc:482] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:19:00.895539 32487 raft_consensus.cc:3037] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:00.896262 32487 raft_consensus.cc:504] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.896396 32487 leader_election.cc:288] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [CANDIDATE]: Term 1 election: Requested vote from peers f2a075f25fef40aba5732a8f6187c278 (127.13.160.129:39813), e932c446244947bda9960388cb7497a7 (127.13.160.130:33911)
I20260925 05:19:00.896538 32150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "887b58ba86594c548cfa1d202df18308" candidate_uuid: "55dce3cdb4204cce831c485bb56f14be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a075f25fef40aba5732a8f6187c278"
I20260925 05:19:00.896560 32284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "887b58ba86594c548cfa1d202df18308" candidate_uuid: "55dce3cdb4204cce831c485bb56f14be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e932c446244947bda9960388cb7497a7"
I20260925 05:19:00.896629 32150 raft_consensus.cc:3037] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:00.896629 32284 raft_consensus.cc:3037] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:00.897557 32284 raft_consensus.cc:2454] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55dce3cdb4204cce831c485bb56f14be in term 1.
I20260925 05:19:00.897557 32150 raft_consensus.cc:2454] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55dce3cdb4204cce831c485bb56f14be in term 1.
I20260925 05:19:00.897734 32351 leader_election.cc:302] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55dce3cdb4204cce831c485bb56f14be, e932c446244947bda9960388cb7497a7; no voters:
I20260925 05:19:00.897832 32487 raft_consensus.cc:2781] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:00.897998 32487 raft_consensus.cc:686] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 1 LEADER]: Becoming Leader. State: Replica: 55dce3cdb4204cce831c485bb56f14be, State: Running, Role: LEADER
I20260925 05:19:00.898108 32487 consensus_queue.cc:237] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } }
I20260925 05:19:00.899286 32005 catalog_manager.cc:5495] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be reported cstate change: term changed from 0 to 1, leader changed from <none> to 55dce3cdb4204cce831c485bb56f14be (127.13.160.131). New cstate: current_term: 1 leader_uuid: "55dce3cdb4204cce831c485bb56f14be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55dce3cdb4204cce831c485bb56f14be" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 45997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 } health_report { overall_health: UNKNOWN } } }
I20260925 05:19:00.905781 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.903271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.13.160.131:45997)
0925 05:19:00.903335 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:00.903337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:00.903346 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:00.903417 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:00.903418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:00.903438 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:00.903527 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:00.903675 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.904160 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.904925 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:00.905141 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:00.905233 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:00.905384 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:00.905487 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:00.905604 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:00.905633 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 05:19:00.905645 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 05:19:00.907783 32284 raft_consensus.cc:1261] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 1 FOLLOWER]: Refusing update from remote peer 55dce3cdb4204cce831c485bb56f14be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:00.907819 32150 raft_consensus.cc:1261] T 887b58ba86594c548cfa1d202df18308 P f2a075f25fef40aba5732a8f6187c278 [term 1 FOLLOWER]: Refusing update from remote peer 55dce3cdb4204cce831c485bb56f14be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:00.908167 32487 consensus_queue.cc:1035] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e932c446244947bda9960388cb7497a7" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:00.908349 32491 consensus_queue.cc:1035] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a075f25fef40aba5732a8f6187c278" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:00.911753 32495 mvcc.cc:205] Tried to move back new op lower bound from 7333124263555395584 to 7333124263519391744. Current Snapshot: MvccSnapshot[applied={T|T < 7333124263555395584}]
I20260925 05:19:00.911846 32496 mvcc.cc:205] Tried to move back new op lower bound from 7333124263555395584 to 7333124263519391744. Current Snapshot: MvccSnapshot[applied={T|T < 7333124263555395584}]
I20260925 05:19:00.914155 32005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:00.914271 32005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:00.915081 32005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 748fd23abe754de79d8c3ad4f9126f8a: Sending DeleteTablet for 3 replicas of tablet 887b58ba86594c548cfa1d202df18308
I20260925 05:19:00.915444 32398 tablet_service.cc:1514] Processing DeleteTablet for tablet 887b58ba86594c548cfa1d202df18308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260925 05:19:00.915510 32130 tablet_service.cc:1514] Processing DeleteTablet for tablet 887b58ba86594c548cfa1d202df18308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:50518
I20260925 05:19:00.915580 32264 tablet_service.cc:1514] Processing DeleteTablet for tablet 887b58ba86594c548cfa1d202df18308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:00 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:45756
I20260925 05:19:00.915791 32505 tablet_replica.cc:330] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: stopping tablet replica
I20260925 05:19:00.915908 32505 raft_consensus.cc:2229] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:19:00.915949 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32067
I20260925 05:19:00.916028 32505 raft_consensus.cc:2258] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:00.916499 32505 ts_tablet_manager.cc:1905] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:00.918728 32505 ts_tablet_manager.cc:1918] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:00.918823 32505 log.cc:1196] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/887b58ba86594c548cfa1d202df18308
I20260925 05:19:00.919132 32505 ts_tablet_manager.cc:1939] T 887b58ba86594c548cfa1d202df18308 P e932c446244947bda9960388cb7497a7: Deleting consensus metadata
I20260925 05:19:00.920117 31989 catalog_manager.cc:4841] TS e932c446244947bda9960388cb7497a7 (127.13.160.130:33911): tablet 887b58ba86594c548cfa1d202df18308 (table test-table [id=a95262a4340f452885e1180e2dec2d5e]) successfully deleted
I20260925 05:19:00.923050 32503 tablet_replica.cc:330] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: stopping tablet replica
I20260925 05:19:00.923226 32503 raft_consensus.cc:2229] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:19:00.923396 32503 raft_consensus.cc:2258] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:00.923982 32503 ts_tablet_manager.cc:1905] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:00.925726 32503 ts_tablet_manager.cc:1918] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:00.925829 32503 log.cc:1196] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/887b58ba86594c548cfa1d202df18308
I20260925 05:19:00.926103 32503 ts_tablet_manager.cc:1939] T 887b58ba86594c548cfa1d202df18308 P 55dce3cdb4204cce831c485bb56f14be: Deleting consensus metadata
I20260925 05:19:00.926692 31991 catalog_manager.cc:4841] TS 55dce3cdb4204cce831c485bb56f14be (127.13.160.131:45997): tablet 887b58ba86594c548cfa1d202df18308 (table test-table [id=a95262a4340f452885e1180e2dec2d5e]) successfully deleted
I20260925 05:19:00.926918 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32201
W20260925 05:19:00.926914 31989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:39813 (error 108)
I20260925 05:19:00.927837 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:00.927697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39813 (local address 127.0.0.1:50524)
0925 05:19:00.927756 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:00.927797 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:39813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 05:19:00.927955 31989 catalog_manager.cc:4584] TS f2a075f25fef40aba5732a8f6187c278 (127.13.160.129:39813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 887b58ba86594c548cfa1d202df18308: Network error: Client connection negotiation failed: client connection to 127.13.160.129:39813: connect: Connection refused (error 111)
I20260925 05:19:00.933005 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32335
I20260925 05:19:00.937944 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 31973
2026-09-25T05:19:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1943 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T05:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:00Z Disabled control of system clock
I20260925 05:19:00.965817 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38519
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:38519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:19:01.048936 32510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:01.049192 32510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:01.049216 32510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:01.050323 32510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:01.050354 32510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:01.050366 32510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:01.050379 32510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:01.050391 32510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:01.051754 32510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:38519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:38519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:01.051961 32510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:01.052211 32510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:01.056982 32516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.056982 32518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.057122 32515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.057358 32510 server_base.cc:1034] running on GCE node
I20260925 05:19:01.057659 32510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:01.058022 32510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:01.059208 32510 hybrid_clock.cc:648] HybridClock initialized: now 1790313541059197 us; error 39 us; skew 500 ppm
I20260925 05:19:01.060782 32510 webserver.cc:460] Webserver started at http://127.13.160.190:46189/ using document root <none> and password file <none>
I20260925 05:19:01.061092 32510 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:01.061174 32510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:01.061338 32510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:01.062512 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "2e522a83bae34623ac72fb1d6345623c"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "3043d023800809d8cb0a4a64dfa5a563"
server_key_iv: "18229d7ee73e9678d84fa5f60f6bdcf6"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.063074 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "2e522a83bae34623ac72fb1d6345623c"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "3043d023800809d8cb0a4a64dfa5a563"
server_key_iv: "18229d7ee73e9678d84fa5f60f6bdcf6"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.064527 32510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:01.065342 32524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.065518 32510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:01.065632 32510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2e522a83bae34623ac72fb1d6345623c"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "3043d023800809d8cb0a4a64dfa5a563"
server_key_iv: "18229d7ee73e9678d84fa5f60f6bdcf6"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.065722 32510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:01.092218 32510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:01.092592 32510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:01.092769 32510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:01.097174 32510 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:38519
I20260925 05:19:01.097236 32576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:38519 every 8 connection(s)
I20260925 05:19:01.097636 32510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:01.098785 32577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:01.100538 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32510
I20260925 05:19:01.100646 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:01.100879 13954 external_mini_cluster.cc:1428] Setting key 1a69fa09aa2223f2e120604ef58f8f49
I20260925 05:19:01.101500 32577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: Bootstrap starting.
I20260925 05:19:01.102136 32577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:01.102499 32577 log.cc:824] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:01.103269 32577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: No bootstrap required, opened a new log
I20260925 05:19:01.105579 32577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } }
I20260925 05:19:01.105792 32577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:01.105832 32577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e522a83bae34623ac72fb1d6345623c, State: Initialized, Role: FOLLOWER
I20260925 05:19:01.105989 32577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } }
I20260925 05:19:01.106087 32577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:01.106127 32577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:01.106168 32577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:01.106854 32577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } }
I20260925 05:19:01.106987 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.101518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41298 (local address 127.13.160.190:38519)
0925 05:19:01.101732 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.101740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.101769 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:19:01.102257 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.102265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.102332 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.102455 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:01.102720 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.103742 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.104862 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.105267 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.105357 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.106166 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:01.106171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:01.106171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:01.106373 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:01.106407 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:01.106419 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:01.106494 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.106607 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.106696 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260925 05:19:01.106986 32577 leader_election.cc:302] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e522a83bae34623ac72fb1d6345623c; no voters:
I20260925 05:19:01.107424 32577 leader_election.cc:288] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:01.107452 32582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:01.107599 32582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [term 1 LEADER]: Becoming Leader. State: Replica: 2e522a83bae34623ac72fb1d6345623c, State: Running, Role: LEADER
I20260925 05:19:01.107686 32582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } }
I20260925 05:19:01.107759 32577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:01.108018 32584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e522a83bae34623ac72fb1d6345623c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } } }
I20260925 05:19:01.108018 32583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e522a83bae34623ac72fb1d6345623c. Latest consensus state: current_term: 1 leader_uuid: "2e522a83bae34623ac72fb1d6345623c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e522a83bae34623ac72fb1d6345623c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 38519 } } }
I20260925 05:19:01.108134 32584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:01.108140 32583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:01.108430 32588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:01.109047 32588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:01.110631 32588 catalog_manager.cc:1347] Generated new cluster ID: 0af2f24934294ab3a5634df630d5edb9
I20260925 05:19:01.110688 32588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:01.129689 32588 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:01.130261 32588 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:01.139793 32588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: Generated new TSK 0
I20260925 05:19:01.140028 32588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:01.147158 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--builtin_ntp_servers=127.13.160.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:19:01.230298 32601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:01.230500 32601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:01.230525 32601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:01.231822 32601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:01.231860 32601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:01.231885 32601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:19:01.233304 32601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:01.233566 32601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:01.233887 32601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:01.238868 32609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.238860 32606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.238999 32601 server_base.cc:1034] running on GCE node
W20260925 05:19:01.238858 32607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.239270 32601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:01.239540 32601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:01.240718 32601 hybrid_clock.cc:648] HybridClock initialized: now 1790313541240699 us; error 30 us; skew 500 ppm
I20260925 05:19:01.241997 32601 webserver.cc:460] Webserver started at http://127.13.160.129:46487/ using document root <none> and password file <none>
I20260925 05:19:01.242268 32601 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:01.242340 32601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:01.242498 32601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:01.243654 32601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "c2ed176a82cd472884921b684e896307"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "45e0c9d05a814f75d99c66d935147bf6"
server_key_iv: "2a8b88341f7bab5603938d54c9bb4d2b"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.244029 32601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "c2ed176a82cd472884921b684e896307"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "45e0c9d05a814f75d99c66d935147bf6"
server_key_iv: "2a8b88341f7bab5603938d54c9bb4d2b"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.245383 32601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:01.246140 32615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.246313 32601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:01.246420 32601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "c2ed176a82cd472884921b684e896307"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "45e0c9d05a814f75d99c66d935147bf6"
server_key_iv: "2a8b88341f7bab5603938d54c9bb4d2b"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.246497 32601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:01.260490 32601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:01.260831 32601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:01.260985 32601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:01.261247 32601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:01.261593 32601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:01.261647 32601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.261690 32601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:01.261715 32601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.267652 32601 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:37227
I20260925 05:19:01.267699 32728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:37227 every 8 connection(s)
I20260925 05:19:01.268036 32601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:19:01.272136 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32601
I20260925 05:19:01.272224 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:19:01.272442 13954 external_mini_cluster.cc:1428] Setting key 6fcae3fa70ab655ff3b64cf31f3e51dc
I20260925 05:19:01.273526 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--builtin_ntp_servers=127.13.160.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 05:19:01.273590 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.268763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:50187 (local address 127.13.160.190:38519)
0925 05:19:01.268831 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.268834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.269055 (+ 221us) server_negotiation.cc:407] Connection header received
0925 05:19:01.269667 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.269668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.269678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.269741 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:01.270410 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.270982 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.272001 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.272226 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.272285 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.273315 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:01.273317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:01.273317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:01.273322 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:01.273340 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:01.273345 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:01.273407 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.273470 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.273497 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 05:19:01.273808 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.268723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38519 (local address 127.13.160.129:50187)
0925 05:19:01.268910 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.268952 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.269559 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.269821 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.269824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.269833 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:01.270332 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.270335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.271098 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.271099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.271890 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.271892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.272324 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.272326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.272457 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.273151 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:01.273152 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:01.273193 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:01.273224 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:01.273408 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:01.273411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:01.273413 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.273456 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.273535 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":109,"threads_started":1}
I20260925 05:19:01.275091 32729 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38519
I20260925 05:19:01.275213 32729 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:01.275401 32729 heartbeater.cc:511] Master 127.13.160.190:38519 requested a full tablet report, sending...
I20260925 05:19:01.275960 32541 ts_manager.cc:194] Registered new tserver with Master: c2ed176a82cd472884921b684e896307 (127.13.160.129:37227)
I20260925 05:19:01.276902 32541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.129:50187
W20260925 05:19:01.354678 32732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:01.354878 32732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:01.354918 32732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:01.355998 32732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:01.356036 32732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:01.356061 32732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:19:01.357342 32732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:01.357527 32732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:01.357771 32732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:01.362643 32740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.362670 32738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.362628 32737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.362919 32732 server_base.cc:1034] running on GCE node
I20260925 05:19:01.363150 32732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:01.363418 32732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:01.364564 32732 hybrid_clock.cc:648] HybridClock initialized: now 1790313541364553 us; error 46 us; skew 500 ppm
I20260925 05:19:01.365764 32732 webserver.cc:460] Webserver started at http://127.13.160.130:40107/ using document root <none> and password file <none>
I20260925 05:19:01.366031 32732 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:01.366099 32732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:01.366266 32732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:01.367384 32732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "d93a210365c0f624dc2689ae5246730c"
server_key_iv: "684a30609c81366eaebb0b04d826ea41"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.367733 32732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "d93a210365c0f624dc2689ae5246730c"
server_key_iv: "684a30609c81366eaebb0b04d826ea41"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.369125 32732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:01.369853 32746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.370002 32732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:01.370105 32732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "d93a210365c0f624dc2689ae5246730c"
server_key_iv: "684a30609c81366eaebb0b04d826ea41"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.370185 32732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:01.406502 32732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:01.406857 32732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:01.407047 32732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:01.407297 32732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:01.407660 32732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:01.407740 32732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.407783 32732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:01.407811 32732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.413672 32732 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38245
I20260925 05:19:01.413714 391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38245 every 8 connection(s)
I20260925 05:19:01.414106 32732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:19:01.419205 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.414943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:53369 (local address 127.13.160.190:38519)
0925 05:19:01.415017 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.415019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.415280 (+ 261us) server_negotiation.cc:407] Connection header received
0925 05:19:01.415752 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.415753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.415762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.415823 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:01.416474 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.416977 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.417934 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.418117 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.418172 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.418966 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:01.418968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:01.418968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:01.418972 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:01.418986 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:01.418990 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:01.419033 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.419118 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.419140 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:19:01.419361 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 32732
I20260925 05:19:01.419483 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:19:01.419488 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.414938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38519 (local address 127.13.160.130:53369)
0925 05:19:01.415135 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.415186 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.415671 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.415876 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.415879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.415887 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:01.416391 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.416395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.417059 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.417059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.417849 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.417850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.418203 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.418204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.418314 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.418818 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:01.418818 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:01.418846 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:01.418870 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:01.419046 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:01.419050 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:01.419052 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.419105 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.419236 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260925 05:19:01.419703 13954 external_mini_cluster.cc:1428] Setting key f3100b294feadc0ef60ca384786c5926
I20260925 05:19:01.420145 392 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38519
I20260925 05:19:01.420241 392 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:01.420439 392 heartbeater.cc:511] Master 127.13.160.190:38519 requested a full tablet report, sending...
I20260925 05:19:01.420801 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--builtin_ntp_servers=127.13.160.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 05:19:01.420866 32541 ts_manager.cc:194] Registered new tserver with Master: fd9e7d9158cc4e8fbfda69348f94b4ab (127.13.160.130:38245)
I20260925 05:19:01.421276 32541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.130:53369
W20260925 05:19:01.501667 395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:01.501876 395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:01.501897 395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:01.503014 395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:01.503049 395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:01.503074 395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:19:01.504331 395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:01.504518 395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:01.504770 395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:01.509794 403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:01.509796 401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.509929 395 server_base.cc:1034] running on GCE node
W20260925 05:19:01.510043 400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:01.510253 395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:01.510497 395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:01.511633 395 hybrid_clock.cc:648] HybridClock initialized: now 1790313541511622 us; error 29 us; skew 500 ppm
I20260925 05:19:01.512832 395 webserver.cc:460] Webserver started at http://127.13.160.131:39827/ using document root <none> and password file <none>
I20260925 05:19:01.513072 395 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:01.513118 395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:01.513262 395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:01.514313 395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "8f8151a6bc162fd04f9ad79aea6e7be9"
server_key_iv: "70ed4ee377ef282729166e9e490dcd1a"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.514676 395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "8f8151a6bc162fd04f9ad79aea6e7be9"
server_key_iv: "70ed4ee377ef282729166e9e490dcd1a"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.516019 395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:01.516721 409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.516913 395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:01.517009 395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9"
format_stamp: "Formatted at 2026-09-25 05:19:01 on dist-test-slave-9tgz"
server_key: "8f8151a6bc162fd04f9ad79aea6e7be9"
server_key_iv: "70ed4ee377ef282729166e9e490dcd1a"
server_key_version: "encryptionkey@0"
I20260925 05:19:01.517089 395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:01.532991 395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:01.533315 395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:01.533464 395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:01.533703 395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:01.534044 395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:01.534104 395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.534149 395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:01.534179 395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:01.539867 395 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:43171
I20260925 05:19:01.539915 524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:43171 every 8 connection(s)
I20260925 05:19:01.540282 395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:19:01.545691 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 395
I20260925 05:19:01.545760 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.541064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:44165 (local address 127.13.160.190:38519)
0925 05:19:01.541111 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.541113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.541371 (+ 258us) server_negotiation.cc:407] Connection header received
0925 05:19:01.541933 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.541935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.541948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.541996 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:01.542728 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.543247 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.544235 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.544430 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.544504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.545398 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:01.545400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:01.545400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:01.545405 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:01.545421 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:01.545425 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:01.545535 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.545640 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.545666 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:01.545823 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:19:01.545960 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.541055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:38519 (local address 127.13.160.131:44165)
0925 05:19:01.541246 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.541279 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.541804 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.542085 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.542089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.542096 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:01.542629 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.542634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.543354 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.543355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.544142 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.544143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.544524 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.544525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.544656 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.545233 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:01.545234 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:01.545269 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:01.545291 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:01.545539 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:01.545546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:01.545549 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.545641 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.545753 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260925 05:19:01.546030 13954 external_mini_cluster.cc:1428] Setting key a5ab7b8c963c05fa65b0fdb0c04451c3
I20260925 05:19:01.546617 525 heartbeater.cc:347] Connected to a master server at 127.13.160.190:38519
I20260925 05:19:01.546718 525 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:01.546952 525 heartbeater.cc:511] Master 127.13.160.190:38519 requested a full tablet report, sending...
I20260925 05:19:01.547328 32540 ts_manager.cc:194] Registered new tserver with Master: f77abe0c3cbe4d9dad25d1cc71e4e3f9 (127.13.160.131:43171)
I20260925 05:19:01.547708 32540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.131:44165
I20260925 05:19:01.558082 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:19:01.562325 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.559621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.13.160.190:38519)
0925 05:19:01.559667 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.559670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.559788 (+ 118us) server_negotiation.cc:407] Connection header received
0925 05:19:01.559827 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.559827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.559837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.559881 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:01.559996 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.560478 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.561203 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.561387 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.561447 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.562032 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:01.562033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:01.562034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:01.562038 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:01.562051 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:01.562055 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:01.562140 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.562181 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.562191 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:01.564086 32540 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:19:01.565675 32540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:19:01.572168 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.568165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59948 (local address 127.13.160.131:43171)
0925 05:19:01.568421 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.568425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.568435 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:19:01.568475 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.568477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.568495 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.568570 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.568755 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.569589 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571166 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.571781 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571864 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.571949 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.572038 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.572074 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":72,"threads_started":1}
I20260925 05:19:01.572152 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.567938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:43171 (local address 127.0.0.1:59948)
0925 05:19:01.568326 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.568337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.568392 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.568628 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.568629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.568632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.568677 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.568678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.569673 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.569675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.571104 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.571106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.571862 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.571937 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.571950 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.572025 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.572075 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":25,"thread_start_us":56,"threads_started":1}
I20260925 05:19:01.572274 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.567939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37227 (local address 127.0.0.1:36716)
0925 05:19:01.568469 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.568477 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.568529 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.568778 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.568779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.568781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.568829 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.568831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.570041 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.570042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.570956 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.570959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.571976 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.572040 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.572043 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.572116 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.572156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":5,"thread_start_us":72,"threads_started":1}
I20260925 05:19:01.572486 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.568146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.13.160.129:37227)
0925 05:19:01.568344 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.568350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.568550 (+ 200us) server_negotiation.cc:407] Connection header received
0925 05:19:01.568601 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.568603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.568620 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.568696 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.568916 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.569954 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571040 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.571778 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571851 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.571940 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.572130 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.572183 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260925 05:19:01.572561 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.567912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38245 (local address 127.0.0.1:43856)
0925 05:19:01.568072 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.568112 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.568175 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.568407 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.568410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.568412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.568482 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.568490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.569922 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.569923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.571021 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.571023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.572344 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.572345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.572405 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.572408 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.572456 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.572495 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260925 05:19:01.572616 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.568009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.13.160.130:38245)
0925 05:19:01.568233 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.568239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.568248 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:01.568285 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.568287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.568301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.568390 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.568588 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.569786 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571092 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.571776 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.571851 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.571940 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.572473 (+ 533us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.572521 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260925 05:19:01.573076 458 tablet_service.cc:1467] Processing CreateTablet for tablet 19bb5eb7dc3c479897370b103c11200b (DEFAULT_TABLE table=test-table [id=20a1f75fb83744de874e584724a1a158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:01.573122 32663 tablet_service.cc:1467] Processing CreateTablet for tablet 19bb5eb7dc3c479897370b103c11200b (DEFAULT_TABLE table=test-table [id=20a1f75fb83744de874e584724a1a158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:01.573468 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bb5eb7dc3c479897370b103c11200b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:01.573469 458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bb5eb7dc3c479897370b103c11200b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:01.576951 540 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: Bootstrap starting.
I20260925 05:19:01.577162 539 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Bootstrap starting.
I20260925 05:19:01.577327 326 tablet_service.cc:1467] Processing CreateTablet for tablet 19bb5eb7dc3c479897370b103c11200b (DEFAULT_TABLE table=test-table [id=20a1f75fb83744de874e584724a1a158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:01.577698 326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bb5eb7dc3c479897370b103c11200b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:01.577836 539 tablet_bootstrap.cc:654] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:01.578126 539 log.cc:824] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:01.578924 540 tablet_bootstrap.cc:654] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:01.578998 539 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: No bootstrap required, opened a new log
I20260925 05:19:01.579070 539 ts_tablet_manager.cc:1397] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:01.579259 540 log.cc:824] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:01.579957 540 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: No bootstrap required, opened a new log
I20260925 05:19:01.580035 540 ts_tablet_manager.cc:1397] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 05:19:01.581089 543 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Bootstrap starting.
I20260925 05:19:01.581324 539 raft_consensus.cc:348] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.581519 539 raft_consensus.cc:374] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:01.581553 539 raft_consensus.cc:729] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77abe0c3cbe4d9dad25d1cc71e4e3f9, State: Initialized, Role: FOLLOWER
I20260925 05:19:01.581712 539 consensus_queue.cc:260] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.581779 543 tablet_bootstrap.cc:654] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:01.582002 539 ts_tablet_manager.cc:1428] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:01.582078 543 log.cc:824] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:01.582142 525 heartbeater.cc:503] Master 127.13.160.190:38519 was elected leader, sending a full tablet report...
I20260925 05:19:01.582551 540 raft_consensus.cc:348] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.582700 540 raft_consensus.cc:374] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:01.582731 540 raft_consensus.cc:729] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ed176a82cd472884921b684e896307, State: Initialized, Role: FOLLOWER
I20260925 05:19:01.582844 540 consensus_queue.cc:260] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.583128 543 tablet_bootstrap.cc:492] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: No bootstrap required, opened a new log
I20260925 05:19:01.583145 540 ts_tablet_manager.cc:1428] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:01.583185 32729 heartbeater.cc:503] Master 127.13.160.190:38519 was elected leader, sending a full tablet report...
I20260925 05:19:01.583196 543 ts_tablet_manager.cc:1397] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:19:01.585218 543 raft_consensus.cc:348] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.585361 543 raft_consensus.cc:374] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:01.585407 543 raft_consensus.cc:729] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9e7d9158cc4e8fbfda69348f94b4ab, State: Initialized, Role: FOLLOWER
I20260925 05:19:01.585516 543 consensus_queue.cc:260] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.585816 543 ts_tablet_manager.cc:1428] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:01.585850 392 heartbeater.cc:503] Master 127.13.160.190:38519 was elected leader, sending a full tablet report...
W20260925 05:19:01.664744 393 tablet.cc:2369] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:19:01.685241 547 raft_consensus.cc:482] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:19:01.685384 547 raft_consensus.cc:504] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.685716 547 leader_election.cc:288] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2ed176a82cd472884921b684e896307 (127.13.160.129:37227), f77abe0c3cbe4d9dad25d1cc71e4e3f9 (127.13.160.131:43171)
I20260925 05:19:01.686223 544 raft_consensus.cc:482] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:19:01.686358 544 raft_consensus.cc:504] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.686718 544 leader_election.cc:288] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd9e7d9158cc4e8fbfda69348f94b4ab (127.13.160.130:38245), c2ed176a82cd472884921b684e896307 (127.13.160.129:37227)
I20260925 05:19:01.689539 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.685942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:43171 (local address 127.13.160.130:60531)
0925 05:19:01.686024 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.686049 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.686101 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.686426 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.686427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.686429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.686500 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.686503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.687430 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.687431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.688632 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.688634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.689312 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.689313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.689378 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.689381 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.689422 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.689461 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260925 05:19:01.689715 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.686032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:56357 (local address 127.13.160.129:37227)
0925 05:19:01.686089 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.686091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.686265 (+ 174us) server_negotiation.cc:407] Connection header received
0925 05:19:01.686302 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.686304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.686319 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.686375 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.686622 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.687353 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.688753 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.689279 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.689342 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.689350 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.689603 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.689640 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:19:01.689844 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.686019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:60531 (local address 127.13.160.131:43171)
0925 05:19:01.686064 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.686066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.686083 (+ 17us) server_negotiation.cc:407] Connection header received
0925 05:19:01.686170 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.686171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.686182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.686258 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.686560 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.687340 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.688716 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.689210 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.689274 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.689282 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.689736 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.689770 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:01.690073 478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" is_pre_election: true
I20260925 05:19:01.690179 478 raft_consensus.cc:2454] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9e7d9158cc4e8fbfda69348f94b4ab in term 0.
I20260925 05:19:01.690313 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.685955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37227 (local address 127.13.160.130:56357)
0925 05:19:01.686164 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.686175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.686248 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.686460 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.686461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.686463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.686538 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.686540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.687438 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.687439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.688649 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.688664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.689391 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.689392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.689454 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.689457 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.689502 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.689550 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260925 05:19:01.690358 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.686849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38245 (local address 127.13.160.131:41941)
0925 05:19:01.686955 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.686964 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.686998 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.687386 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.687387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.687388 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.687452 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.687453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.688372 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.688373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.689497 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.689499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.690134 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.690135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.690213 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.690217 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.690249 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.690285 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260925 05:19:01.690385 32747 leader_election.cc:302] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f77abe0c3cbe4d9dad25d1cc71e4e3f9, fd9e7d9158cc4e8fbfda69348f94b4ab; no voters:
I20260925 05:19:01.690528 547 raft_consensus.cc:2781] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:19:01.690577 547 raft_consensus.cc:482] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:19:01.690596 547 raft_consensus.cc:3037] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:01.690650 32683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ed176a82cd472884921b684e896307" is_pre_election: true
I20260925 05:19:01.690825 32683 raft_consensus.cc:2454] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9e7d9158cc4e8fbfda69348f94b4ab in term 0.
I20260925 05:19:01.691046 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.687227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:41941 (local address 127.13.160.130:38245)
0925 05:19:01.687262 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.687264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.687271 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:19:01.687289 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.687290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.687298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.687516 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.687541 (+ 25us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.688272 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.689582 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.690032 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.690506 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.690514 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.690531 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.690995 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":20}
I20260925 05:19:01.691236 346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" is_pre_election: true
I20260925 05:19:01.691435 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.686922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37227 (local address 127.13.160.131:40437)
0925 05:19:01.687150 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.687160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:01.687203 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:01.687435 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:01.687436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:01.687437 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:01.687495 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.687496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.688425 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.688426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.689546 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:01.689549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.691207 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.691208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:01.691282 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.691286 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:01.691317 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 05:19:01.691353 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":95,"threads_started":1}
I20260925 05:19:01.691593 547 raft_consensus.cc:504] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.691699 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.687109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:40437 (local address 127.13.160.129:37227)
0925 05:19:01.687306 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.687307 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.687314 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:19:01.687333 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.687334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.687343 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.687551 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:01.687581 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.688330 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.690030 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.691097 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.691553 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.691563 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.691583 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.691621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260925 05:19:01.691717 346 raft_consensus.cc:2379] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f77abe0c3cbe4d9dad25d1cc71e4e3f9 in current term 1: Already voted for candidate fd9e7d9158cc4e8fbfda69348f94b4ab in this term.
I20260925 05:19:01.691753 547 leader_election.cc:288] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [CANDIDATE]: Term 1 election: Requested vote from peers c2ed176a82cd472884921b684e896307 (127.13.160.129:37227), f77abe0c3cbe4d9dad25d1cc71e4e3f9 (127.13.160.131:43171)
I20260925 05:19:01.691887 478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9"
I20260925 05:19:01.691884 32683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ed176a82cd472884921b684e896307" is_pre_election: true
I20260925 05:19:01.691954 478 raft_consensus.cc:3037] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:01.691967 32683 raft_consensus.cc:2454] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f77abe0c3cbe4d9dad25d1cc71e4e3f9 in term 0.
I20260925 05:19:01.692198 413 leader_election.cc:302] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c2ed176a82cd472884921b684e896307, f77abe0c3cbe4d9dad25d1cc71e4e3f9; no voters: fd9e7d9158cc4e8fbfda69348f94b4ab
I20260925 05:19:01.692281 32683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bb5eb7dc3c479897370b103c11200b" candidate_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ed176a82cd472884921b684e896307"
I20260925 05:19:01.692348 32683 raft_consensus.cc:3037] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:01.692770 478 raft_consensus.cc:2454] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9e7d9158cc4e8fbfda69348f94b4ab in term 1.
I20260925 05:19:01.692888 544 raft_consensus.cc:2741] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 05:19:01.692955 32747 leader_election.cc:302] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f77abe0c3cbe4d9dad25d1cc71e4e3f9, fd9e7d9158cc4e8fbfda69348f94b4ab; no voters:
I20260925 05:19:01.693034 547 raft_consensus.cc:2781] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:01.693207 547 raft_consensus.cc:686] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 LEADER]: Becoming Leader. State: Replica: fd9e7d9158cc4e8fbfda69348f94b4ab, State: Running, Role: LEADER
I20260925 05:19:01.693177 32683 raft_consensus.cc:2454] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9e7d9158cc4e8fbfda69348f94b4ab in term 1.
I20260925 05:19:01.693311 547 consensus_queue.cc:237] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } }
I20260925 05:19:01.694124 32540 catalog_manager.cc:5495] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab reported cstate change: term changed from 0 to 1, leader changed from <none> to fd9e7d9158cc4e8fbfda69348f94b4ab (127.13.160.130). New cstate: current_term: 1 leader_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e7d9158cc4e8fbfda69348f94b4ab" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 } health_report { overall_health: UNKNOWN } } }
I20260925 05:19:01.723582 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.721268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.13.160.130:38245)
0925 05:19:01.721335 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:01.721337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:01.721347 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:19:01.721386 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:01.721386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:01.721405 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:01.721461 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:01.721586 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.722052 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.722782 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:01.722988 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:01.723042 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:01.723196 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:01.723285 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:01.723355 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:01.723446 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 05:19:01.723460 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:19:01.725520 478 raft_consensus.cc:1261] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 1 FOLLOWER]: Refusing update from remote peer fd9e7d9158cc4e8fbfda69348f94b4ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:01.725519 32683 raft_consensus.cc:1261] T 19bb5eb7dc3c479897370b103c11200b P c2ed176a82cd472884921b684e896307 [term 1 FOLLOWER]: Refusing update from remote peer fd9e7d9158cc4e8fbfda69348f94b4ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:01.725883 547 consensus_queue.cc:1035] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ed176a82cd472884921b684e896307" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 37227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:01.726024 551 consensus_queue.cc:1035] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77abe0c3cbe4d9dad25d1cc71e4e3f9" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:01.728556 554 mvcc.cc:205] Tried to move back new op lower bound from 7333124266904821760 to 7333124266776293376. Current Snapshot: MvccSnapshot[applied={T|T < 7333124266904821760}]
I20260925 05:19:01.728652 556 mvcc.cc:205] Tried to move back new op lower bound from 7333124266904821760 to 7333124266776293376. Current Snapshot: MvccSnapshot[applied={T|T < 7333124266904821760}]
I20260925 05:19:01.731489 32541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:01.731583 32541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:01.732378 32541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e522a83bae34623ac72fb1d6345623c: Sending DeleteTablet for 3 replicas of tablet 19bb5eb7dc3c479897370b103c11200b
I20260925 05:19:01.732674 326 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bb5eb7dc3c479897370b103c11200b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:01 UTC) from {username='slave'} at 127.0.0.1:43856
I20260925 05:19:01.732821 32663 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bb5eb7dc3c479897370b103c11200b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:01 UTC) from {username='slave'} at 127.0.0.1:36716
I20260925 05:19:01.732832 458 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bb5eb7dc3c479897370b103c11200b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:01 UTC) from {username='slave'} at 127.0.0.1:59948
I20260925 05:19:01.733142 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32601
I20260925 05:19:01.733052 565 tablet_replica.cc:330] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: stopping tablet replica
I20260925 05:19:01.735548 565 raft_consensus.cc:2229] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:19:01.735688 565 raft_consensus.cc:2258] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:01.736169 565 ts_tablet_manager.cc:1905] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:01.738211 565 ts_tablet_manager.cc:1918] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:01.738288 565 log.cc:1196] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/19bb5eb7dc3c479897370b103c11200b
I20260925 05:19:01.738530 565 ts_tablet_manager.cc:1939] T 19bb5eb7dc3c479897370b103c11200b P f77abe0c3cbe4d9dad25d1cc71e4e3f9: Deleting consensus metadata
I20260925 05:19:01.739051 564 tablet_replica.cc:330] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: stopping tablet replica
I20260925 05:19:01.739166 564 raft_consensus.cc:2229] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:19:01.739305 564 raft_consensus.cc:2258] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:01.739513 32525 catalog_manager.cc:4841] TS f77abe0c3cbe4d9dad25d1cc71e4e3f9 (127.13.160.131:43171): tablet 19bb5eb7dc3c479897370b103c11200b (table test-table [id=20a1f75fb83744de874e584724a1a158]) successfully deleted
W20260925 05:19:01.739933 32528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:37227 (error 108)
I20260925 05:19:01.739809 564 ts_tablet_manager.cc:1905] T 19bb5eb7dc3c479897370b103c11200b P fd9e7d9158cc4e8fbfda69348f94b4ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:01.740178 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32732
I20260925 05:19:01.742527 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.742369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:37227 (local address 127.0.0.1:36728)
0925 05:19:01.742434 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.742484 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:37227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 05:19:01.742720 32528 catalog_manager.cc:4584] TS c2ed176a82cd472884921b684e896307 (127.13.160.129:37227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19bb5eb7dc3c479897370b103c11200b: Network error: Client connection negotiation failed: client connection to 127.13.160.129:37227: connect: Connection refused (error 111)
I20260925 05:19:01.745914 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 395
I20260925 05:19:01.745980 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:01.745850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38245 (local address 127.0.0.1:43862)
0925 05:19:01.745904 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:01.745936 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 05:19:01.750191 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 32510
2026-09-25T05:19:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (811 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:01 dist-test-slave-9tgz krb5kdc[570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:01 dist-test-slave-9tgz krb5kdc[570](info): set up 2 sockets
Sep 25 05:19:01 dist-test-slave-9tgz krb5kdc[570](info): commencing operation
krb5kdc: starting...
W20260925 05:19:02.775770 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:02 dist-test-slave-9tgz krb5kdc[570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:02.893913 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36317
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:36317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:02.975981 583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:02.976245 583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:02.976281 583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:02.977464 583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:02.977535 583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:02.977560 583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:02.977589 583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:02.977623 583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:02.979143 583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:36317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:02.979437 583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:02.979673 583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:02.984577 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:02.984532 591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:02.984514 589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:02.984730 583 server_base.cc:1034] running on GCE node
I20260925 05:19:02.985113 583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:02.985436 583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:02.986604 583 hybrid_clock.cc:648] HybridClock initialized: now 1790313542986591 us; error 45 us; skew 500 ppm
Sep 25 05:19:02 dist-test-slave-9tgz krb5kdc[570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313542, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:02.989212 583 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:02.989655 583 webserver.cc:460] Webserver started at http://127.13.160.190:36669/ using document root <none> and password file <none>
I20260925 05:19:02.989882 583 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:02.989949 583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:02.990096 583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:02.991271 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "9f4dd532dd514d81a98c4b66e2352ac3"
format_stamp: "Formatted at 2026-09-25 05:19:02 on dist-test-slave-9tgz"
server_key: "4abdb62fcbe5e65fe32129529b5e3723"
server_key_iv: "bdb05f2261de59a5d8bfa937d2851682"
server_key_version: "encryptionkey@0"
I20260925 05:19:02.991627 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "9f4dd532dd514d81a98c4b66e2352ac3"
format_stamp: "Formatted at 2026-09-25 05:19:02 on dist-test-slave-9tgz"
server_key: "4abdb62fcbe5e65fe32129529b5e3723"
server_key_iv: "bdb05f2261de59a5d8bfa937d2851682"
server_key_version: "encryptionkey@0"
I20260925 05:19:02.992973 583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:02.993683 598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:02.993842 583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:02.993947 583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "9f4dd532dd514d81a98c4b66e2352ac3"
format_stamp: "Formatted at 2026-09-25 05:19:02 on dist-test-slave-9tgz"
server_key: "4abdb62fcbe5e65fe32129529b5e3723"
server_key_iv: "bdb05f2261de59a5d8bfa937d2851682"
server_key_version: "encryptionkey@0"
I20260925 05:19:02.994023 583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.002841 583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.004801 583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.004952 583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.008936 583 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:36317
I20260925 05:19:03.008989 651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:36317 every 8 connection(s)
I20260925 05:19:03.009358 583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:03.010233 652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:03.012621 652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: Bootstrap starting.
I20260925 05:19:03.013221 652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:03.013509 652 log.cc:824] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:03.014209 652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: No bootstrap required, opened a new log
I20260925 05:19:03.015888 652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } }
I20260925 05:19:03.016054 652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:03.016103 652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4dd532dd514d81a98c4b66e2352ac3, State: Initialized, Role: FOLLOWER
I20260925 05:19:03.016275 652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } }
I20260925 05:19:03.016360 652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:03.016405 652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:03.016454 652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:03.016983 652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } }
I20260925 05:19:03.017102 652 leader_election.cc:302] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f4dd532dd514d81a98c4b66e2352ac3; no voters:
I20260925 05:19:03.017287 652 leader_election.cc:288] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:03.017333 655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:03.017541 655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 9f4dd532dd514d81a98c4b66e2352ac3, State: Running, Role: LEADER
I20260925 05:19:03.017544 652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:03.017640 655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } }
I20260925 05:19:03.018194 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } } }
I20260925 05:19:03.018334 657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f4dd532dd514d81a98c4b66e2352ac3. Latest consensus state: current_term: 1 leader_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f4dd532dd514d81a98c4b66e2352ac3" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36317 } } }
I20260925 05:19:03.018396 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:03.018414 657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:03.018720 666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:03.019066 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 583
I20260925 05:19:03.019186 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:03.019311 666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:03.019382 13954 external_mini_cluster.cc:1428] Setting key 60979c05e1cfcc75c90b0378b1741d09
I20260925 05:19:03.021064 666 catalog_manager.cc:1347] Generated new cluster ID: 83a3cdc2e2ba4f9e9a7e1e87b2359b75
I20260925 05:19:03.021122 666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:03.031376 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.019873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.13.160.190:36317)
0925 05:19:03.020040 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.020050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.020246 (+ 196us) server_negotiation.cc:407] Connection header received
0925 05:19:03.020393 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.020399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.020443 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.020597 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.021469 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.022870 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.023935 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.024586 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.024676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.027627 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.027632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.027632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:03.027645 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.029639 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.030042 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.030045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.030046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.030155 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.030363 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.030364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.030364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.030686 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.030820 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.030980 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.031122 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260925 05:19:03.042655 666 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:03.043308 666 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:03.056696 666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: Generated new TSK 0
I20260925 05:19:03.056948 666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.160.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
Entry for principal kudu/127.13.160.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab.
I20260925 05:19:03.081671 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:03.165832 678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:03.166110 678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:03.166170 678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:03.167541 678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:03.167614 678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:03.167663 678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:19:03.169395 678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.129.keytab
--principal=kudu/127.13.160.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:03.169725 678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:03.169991 678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:03.175455 683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.175491 686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.175463 684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:03.175760 678 server_base.cc:1034] running on GCE node
I20260925 05:19:03.175993 678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:03.176251 678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:03.177413 678 hybrid_clock.cc:648] HybridClock initialized: now 1790313543177388 us; error 41 us; skew 500 ppm
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:03.180044 678 init.cc:377] Logged in from keytab as kudu/127.13.160.129@KRBTEST.COM (short username kudu)
I20260925 05:19:03.180572 678 webserver.cc:460] Webserver started at http://127.13.160.129:44201/ using document root <none> and password file <none>
I20260925 05:19:03.180822 678 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:03.180887 678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:03.181043 678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:03.182152 678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "3ef3839360e24db9a104806f24abeb37"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "2fdb37d86cbf34273e7c195c6ce3143a"
server_key_iv: "73a85caf13378bb4ee89ce1adb8f964d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.182533 678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "3ef3839360e24db9a104806f24abeb37"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "2fdb37d86cbf34273e7c195c6ce3143a"
server_key_iv: "73a85caf13378bb4ee89ce1adb8f964d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.184049 678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:03.184816 693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.184999 678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:03.185093 678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "3ef3839360e24db9a104806f24abeb37"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "2fdb37d86cbf34273e7c195c6ce3143a"
server_key_iv: "73a85caf13378bb4ee89ce1adb8f964d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.185168 678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.208750 678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.210843 678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.211033 678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.211246 678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:03.211580 678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:03.211612 678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.211637 678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:03.211685 678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.218248 678 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:39247
I20260925 05:19:03.218335 807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:39247 every 8 connection(s)
I20260925 05:19:03.218732 678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.129@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:03.227519 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 678
I20260925 05:19:03.227624 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:19:03.227883 13954 external_mini_cluster.cc:1428] Setting key 05f11df246951e0d1456337646c93e10
I20260925 05:19:03.229286 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.219583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:32813 (local address 127.13.160.190:36317)
0925 05:19:03.219660 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.219662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.219883 (+ 221us) server_negotiation.cc:407] Connection header received
0925 05:19:03.220395 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.220396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.220411 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.220472 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.221458 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.221969 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.223165 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.223363 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.223422 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.226079 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.226081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.226081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:03.226088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.227564 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.228176 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.228178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.228178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.228247 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.228529 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.228530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.228530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.228700 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.228782 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.229087 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.229187 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 05:19:03.229559 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.219564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36317 (local address 127.13.160.129:32813)
0925 05:19:03.219738 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.219784 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.220303 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.220547 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.220551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.221141 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:03.221359 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.221364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.222084 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.222085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.223066 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.223068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.223442 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.223443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.223548 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.224222 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:03.224234 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:03.225983 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:03.227729 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.227740 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.227743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.228085 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.228317 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.228318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.228318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.228451 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.228773 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:03.228781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:03.228945 (+ 164us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.229065 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.229258 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260925 05:19:03.230084 808 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36317
I20260925 05:19:03.230221 808 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:03.230444 808 heartbeater.cc:511] Master 127.13.160.190:36317 requested a full tablet report, sending...
I20260925 05:19:03.231091 615 ts_manager.cc:194] Registered new tserver with Master: 3ef3839360e24db9a104806f24abeb37 (127.13.160.129:39247)
I20260925 05:19:03.231738 615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.129@KRBTEST.COM'} at 127.13.160.129:32813
WARNING: no policy specified for kudu/127.13.160.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
Entry for principal kudu/127.13.160.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab.
I20260925 05:19:03.254194 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:03.348937 813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:03.349170 813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:03.349210 813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:03.350383 813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:03.350452 813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:03.350498 813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:19:03.352375 813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.130.keytab
--principal=kudu/127.13.160.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:03.352645 813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:03.352891 813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:03.357911 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.357929 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.358218 821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:03.358229 813 server_base.cc:1034] running on GCE node
I20260925 05:19:03.358464 813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:03.358673 813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:03.359817 813 hybrid_clock.cc:648] HybridClock initialized: now 1790313543359804 us; error 28 us; skew 500 ppm
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:03.365501 813 init.cc:377] Logged in from keytab as kudu/127.13.160.130@KRBTEST.COM (short username kudu)
I20260925 05:19:03.365979 813 webserver.cc:460] Webserver started at http://127.13.160.130:45705/ using document root <none> and password file <none>
I20260925 05:19:03.366245 813 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:03.366317 813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:03.366469 813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:03.367614 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "810925f270b67337b00c4e41886aeb05"
server_key_iv: "3586e821fb61942af481be01895abf14"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.367985 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "810925f270b67337b00c4e41886aeb05"
server_key_iv: "3586e821fb61942af481be01895abf14"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.369350 813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:03.370090 828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.370276 813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:03.370376 813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "810925f270b67337b00c4e41886aeb05"
server_key_iv: "3586e821fb61942af481be01895abf14"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.370456 813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.380434 813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.382629 813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.382740 813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.383039 813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:03.383311 813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:03.383344 813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.383366 813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:03.383379 813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.389240 813 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:38353
I20260925 05:19:03.389365 942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:38353 every 8 connection(s)
I20260925 05:19:03.389647 813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.130@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:03.398594 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.390320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:60069 (local address 127.13.160.190:36317)
0925 05:19:03.390367 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.390369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.390600 (+ 231us) server_negotiation.cc:407] Connection header received
0925 05:19:03.391129 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.391130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.391140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.391203 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.392115 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.392643 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.393630 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.393807 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.393856 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.395919 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.395921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.395921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:03.395927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.397468 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.397843 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.397844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.397845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.397897 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.398095 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.398095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.398096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.398196 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.398245 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.398468 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.398529 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:19:03.398813 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.390296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36317 (local address 127.13.160.130:60069)
0925 05:19:03.390455 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.390496 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.391042 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.391261 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.391264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.391823 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:03.392019 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.392023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.392731 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.392732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.393551 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.393553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.393879 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.393880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.393976 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.394447 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:03.394455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:03.395830 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:03.397566 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.397571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.397573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.397783 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.397952 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.397953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.397953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.398044 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.398248 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:03.398252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:03.398362 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.398444 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.398557 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260925 05:19:03.399377 943 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36317
I20260925 05:19:03.399488 943 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:03.399636 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 813
I20260925 05:19:03.399700 943 heartbeater.cc:511] Master 127.13.160.190:36317 requested a full tablet report, sending...
I20260925 05:19:03.399750 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:19:03.399946 13954 external_mini_cluster.cc:1428] Setting key ab230fd85a9c591d9a26646ba240c12f
I20260925 05:19:03.400100 615 ts_manager.cc:194] Registered new tserver with Master: a9d3d4f31ee44c7085a8b72f3a47e2d3 (127.13.160.130:38353)
I20260925 05:19:03.400697 615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.130@KRBTEST.COM'} at 127.13.160.130:60069
WARNING: no policy specified for kudu/127.13.160.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
Entry for principal kudu/127.13.160.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab.
I20260925 05:19:03.425153 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:03.506368 948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:03.506618 948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:03.506660 948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:03.507860 948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:03.507930 948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:03.507977 948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:19:03.509416 948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.131.keytab
--principal=kudu/127.13.160.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:36317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:03.509689 948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:03.509927 948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:03.514972 956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.515056 953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.514972 954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:03.515228 948 server_base.cc:1034] running on GCE node
I20260925 05:19:03.515455 948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:03.515722 948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:03.516865 948 hybrid_clock.cc:648] HybridClock initialized: now 1790313543516856 us; error 28 us; skew 500 ppm
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:03.519661 948 init.cc:377] Logged in from keytab as kudu/127.13.160.131@KRBTEST.COM (short username kudu)
I20260925 05:19:03.521231 948 webserver.cc:460] Webserver started at http://127.13.160.131:44745/ using document root <none> and password file <none>
I20260925 05:19:03.521414 948 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:03.521454 948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:03.521548 948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:03.522558 948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "d61acd1f9856438fbbe63a3a4b19d878"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "997b5080f445dd1341ce141141db1c1f"
server_key_iv: "503bb55d6301a2a973eefdd17737035d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.522864 948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "d61acd1f9856438fbbe63a3a4b19d878"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "997b5080f445dd1341ce141141db1c1f"
server_key_iv: "503bb55d6301a2a973eefdd17737035d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.524194 948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:03.524881 963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.525085 948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:03.525157 948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "d61acd1f9856438fbbe63a3a4b19d878"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "997b5080f445dd1341ce141141db1c1f"
server_key_iv: "503bb55d6301a2a973eefdd17737035d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.525209 948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.544909 948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.547374 948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.547489 948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.547696 948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:03.548038 948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:03.548074 948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.548122 948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:03.548149 948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.554142 948 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:41873
I20260925 05:19:03.554193 1078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:41873 every 8 connection(s)
I20260925 05:19:03.554517 948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:19:03.560285 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 948
I20260925 05:19:03.560401 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:19:03.560587 13954 external_mini_cluster.cc:1428] Setting key b3517aaade6ff7396be43e3b6bf13635
Sep 25 05:19:03 dist-test-slave-9tgz krb5kdc[570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313543, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.131@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:03.564275 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.555242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:43557 (local address 127.13.160.190:36317)
0925 05:19:03.555294 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.555309 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.555510 (+ 201us) server_negotiation.cc:407] Connection header received
0925 05:19:03.556060 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.556062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.556074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.556128 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.557308 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.557802 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.558827 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.559078 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.559136 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.561400 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.561402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.561403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:03.561410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.562836 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.563290 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.563292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.563292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.563352 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.563609 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.563610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.563610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.563726 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.563805 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.564093 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.564181 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:19:03.564478 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.555212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:36317 (local address 127.13.160.131:43557)
0925 05:19:03.555374 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.555419 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.555940 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.556213 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.556217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.556960 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:03.557201 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.557207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.557893 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.557894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.558743 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.558745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.559155 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.559156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.559261 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.559727 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:03.559735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:03.561305 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:03.562958 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.562965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.562968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.563201 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.563421 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:03.563421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:03.563422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:03.563541 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:03.563794 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:03.563800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:03.563941 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.564068 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.564239 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260925 05:19:03.564958 1079 heartbeater.cc:347] Connected to a master server at 127.13.160.190:36317
I20260925 05:19:03.565065 1079 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:03.565243 1079 heartbeater.cc:511] Master 127.13.160.190:36317 requested a full tablet report, sending...
I20260925 05:19:03.565625 615 ts_manager.cc:194] Registered new tserver with Master: d61acd1f9856438fbbe63a3a4b19d878 (127.13.160.131:41873)
I20260925 05:19:03.565969 615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.160.131@KRBTEST.COM'} at 127.13.160.131:43557
I20260925 05:19:03.572372 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:19:03.579567 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.573688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.13.160.190:36317)
0925 05:19:03.573732 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.573734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.573895 (+ 161us) server_negotiation.cc:407] Connection header received
0925 05:19:03.573926 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.573926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.573937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.573982 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.574489 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.575082 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.575862 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.576043 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.576095 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.577183 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.577184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.577184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:03.577190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.578567 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.578861 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.578862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.578862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.578959 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:03.579104 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:03.579105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:03.579105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:03.579245 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.579319 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.579407 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.579462 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:19:03.580972 615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:19:03.582664 615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:19:03.589318 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.585194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38353 (local address 127.0.0.1:41082)
0925 05:19:03.585490 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.585502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.585583 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.585944 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.585945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.585948 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:03.585987 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.585988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.587233 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.587234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.588060 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.588062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.588994 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.588995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.589087 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589103 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.589170 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.589231 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":67,"threads_started":1}
I20260925 05:19:03.589586 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.585107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.13.160.131:41873)
0925 05:19:03.585515 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.585520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.585532 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:19:03.585573 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.585576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.585593 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.585692 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:03.585869 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.586641 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.588308 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.589103 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.589378 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589440 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.589473 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.589510 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":100,"threads_started":1}
I20260925 05:19:03.589401 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.585326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.13.160.129:39247)
0925 05:19:03.585505 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.585509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.585519 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:19:03.585556 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.585558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.585575 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.585648 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:03.585889 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.587037 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.588307 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.589039 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.589125 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589185 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.589258 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.589309 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260925 05:19:03.589610 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.585318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.13.160.130:38353)
0925 05:19:03.585488 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.585493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.585806 (+ 313us) server_negotiation.cc:407] Connection header received
0925 05:19:03.585851 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.585853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.585870 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.585948 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:03.586054 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.587123 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.588164 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.588891 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.588993 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589052 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.589506 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.589554 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260925 05:19:03.589434 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.584977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:41873 (local address 127.0.0.1:52436)
0925 05:19:03.585182 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.585224 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.585288 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.585699 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.585701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.585704 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:03.585788 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.585790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.586746 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.586747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.588197 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.588200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.589192 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.589193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.589259 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589264 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.589294 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.589333 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":58,"threads_started":1}
I20260925 05:19:03.589331 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.584995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39247 (local address 127.0.0.1:48500)
0925 05:19:03.585324 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.585334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.585373 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.585749 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.585750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.585753 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:03.585803 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.585805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.587141 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.587142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.588220 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.588221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.589120 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.589121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.589183 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.589187 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.589243 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.589271 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":117,"threads_started":1}
I20260925 05:19:03.590478 1012 tablet_service.cc:1467] Processing CreateTablet for tablet 850a48e4b88b4aefa72acc682a841540 (DEFAULT_TABLE table=test-table [id=fd569c01685246cba6a631de1e84d4b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:03.590551 876 tablet_service.cc:1467] Processing CreateTablet for tablet 850a48e4b88b4aefa72acc682a841540 (DEFAULT_TABLE table=test-table [id=fd569c01685246cba6a631de1e84d4b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:03.590548 741 tablet_service.cc:1467] Processing CreateTablet for tablet 850a48e4b88b4aefa72acc682a841540 (DEFAULT_TABLE table=test-table [id=fd569c01685246cba6a631de1e84d4b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:03.590852 1012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a48e4b88b4aefa72acc682a841540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:03.590858 741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a48e4b88b4aefa72acc682a841540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:03.590858 876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a48e4b88b4aefa72acc682a841540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:03.593783 1096 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Bootstrap starting.
I20260925 05:19:03.594295 1097 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: Bootstrap starting.
I20260925 05:19:03.594472 1096 tablet_bootstrap.cc:654] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:03.594766 1097 tablet_bootstrap.cc:654] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:03.595038 1096 log.cc:824] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:03.595070 1098 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Bootstrap starting.
I20260925 05:19:03.595322 1097 log.cc:824] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:03.595690 1098 tablet_bootstrap.cc:654] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:03.595798 1096 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: No bootstrap required, opened a new log
I20260925 05:19:03.595872 1096 ts_tablet_manager.cc:1397] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:03.596175 1097 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: No bootstrap required, opened a new log
I20260925 05:19:03.596167 1098 log.cc:824] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:03.596282 1097 ts_tablet_manager.cc:1397] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 05:19:03.596969 1098 tablet_bootstrap.cc:492] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: No bootstrap required, opened a new log
I20260925 05:19:03.597057 1098 ts_tablet_manager.cc:1397] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:03.597886 1096 raft_consensus.cc:348] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.598076 1096 raft_consensus.cc:374] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:03.598114 1096 raft_consensus.cc:729] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61acd1f9856438fbbe63a3a4b19d878, State: Initialized, Role: FOLLOWER
I20260925 05:19:03.598254 1096 consensus_queue.cc:260] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.598345 1097 raft_consensus.cc:348] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.598516 1097 raft_consensus.cc:374] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:03.598527 1096 ts_tablet_manager.cc:1428] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:03.598560 1097 raft_consensus.cc:729] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef3839360e24db9a104806f24abeb37, State: Initialized, Role: FOLLOWER
I20260925 05:19:03.598649 1079 heartbeater.cc:503] Master 127.13.160.190:36317 was elected leader, sending a full tablet report...
I20260925 05:19:03.598676 1097 consensus_queue.cc:260] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.598968 1097 ts_tablet_manager.cc:1428] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 05:19:03.599038 808 heartbeater.cc:503] Master 127.13.160.190:36317 was elected leader, sending a full tablet report...
I20260925 05:19:03.599143 1098 raft_consensus.cc:348] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.599289 1098 raft_consensus.cc:374] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:03.599330 1098 raft_consensus.cc:729] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9d3d4f31ee44c7085a8b72f3a47e2d3, State: Initialized, Role: FOLLOWER
I20260925 05:19:03.599445 1098 consensus_queue.cc:260] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.599774 943 heartbeater.cc:503] Master 127.13.160.190:36317 was elected leader, sending a full tablet report...
I20260925 05:19:03.599722 1098 ts_tablet_manager.cc:1428] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:03.617194 1102 raft_consensus.cc:482] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:19:03.617352 1102 raft_consensus.cc:504] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.617794 1102 leader_election.cc:288] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9d3d4f31ee44c7085a8b72f3a47e2d3 (127.13.160.130:38353), 3ef3839360e24db9a104806f24abeb37 (127.13.160.129:39247)
I20260925 05:19:03.621819 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.618078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:36195 (local address 127.13.160.129:39247)
0925 05:19:03.618178 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.618180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.618334 (+ 154us) server_negotiation.cc:407] Connection header received
0925 05:19:03.618375 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.618376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.618386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.618429 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:03.618621 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.619352 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.620854 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.621439 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.621517 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.621526 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.621708 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.621749 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 05:19:03.621819 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.618059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:57105 (local address 127.13.160.130:38353)
0925 05:19:03.618110 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.618112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.618123 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:03.618216 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.618218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.618239 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.618281 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:03.618530 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.619305 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.620854 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.621439 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.621517 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.621526 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.621708 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.621747 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:19:03.621860 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.617967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38353 (local address 127.13.160.131:57105)
0925 05:19:03.618057 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.618069 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.618111 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.618310 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.618311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.618313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:03.618406 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.618408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.619410 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.619412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.620755 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.620758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.621534 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.621535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.621623 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.621626 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.621689 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.621743 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":74}
I20260925 05:19:03.621869 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.618045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39247 (local address 127.13.160.131:36195)
0925 05:19:03.618241 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.618249 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.618324 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.618450 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.618451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.618452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:03.618520 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.618521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.619423 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.619423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.620755 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.620758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.621534 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.621535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.621618 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.621622 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.621689 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.621743 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":60,"thread_start_us":115,"threads_started":1}
I20260925 05:19:03.622246 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a48e4b88b4aefa72acc682a841540" candidate_uuid: "d61acd1f9856438fbbe63a3a4b19d878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" is_pre_election: true
I20260925 05:19:03.622421 896 raft_consensus.cc:2454] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61acd1f9856438fbbe63a3a4b19d878 in term 0.
I20260925 05:19:03.622398 761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a48e4b88b4aefa72acc682a841540" candidate_uuid: "d61acd1f9856438fbbe63a3a4b19d878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef3839360e24db9a104806f24abeb37" is_pre_election: true
I20260925 05:19:03.622596 761 raft_consensus.cc:2454] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61acd1f9856438fbbe63a3a4b19d878 in term 0.
I20260925 05:19:03.622634 967 leader_election.cc:302] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9d3d4f31ee44c7085a8b72f3a47e2d3, d61acd1f9856438fbbe63a3a4b19d878; no voters:
I20260925 05:19:03.622761 1102 raft_consensus.cc:2781] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:19:03.622807 1102 raft_consensus.cc:482] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:19:03.622864 1102 raft_consensus.cc:3037] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:03.623607 1102 raft_consensus.cc:504] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.623766 1102 leader_election.cc:288] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [CANDIDATE]: Term 1 election: Requested vote from peers a9d3d4f31ee44c7085a8b72f3a47e2d3 (127.13.160.130:38353), 3ef3839360e24db9a104806f24abeb37 (127.13.160.129:39247)
I20260925 05:19:03.623888 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a48e4b88b4aefa72acc682a841540" candidate_uuid: "d61acd1f9856438fbbe63a3a4b19d878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3"
I20260925 05:19:03.623948 761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a48e4b88b4aefa72acc682a841540" candidate_uuid: "d61acd1f9856438fbbe63a3a4b19d878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef3839360e24db9a104806f24abeb37"
I20260925 05:19:03.623965 896 raft_consensus.cc:3037] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:03.624017 761 raft_consensus.cc:3037] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:03.624876 896 raft_consensus.cc:2454] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61acd1f9856438fbbe63a3a4b19d878 in term 1.
I20260925 05:19:03.624874 761 raft_consensus.cc:2454] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61acd1f9856438fbbe63a3a4b19d878 in term 1.
I20260925 05:19:03.625062 967 leader_election.cc:302] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9d3d4f31ee44c7085a8b72f3a47e2d3, d61acd1f9856438fbbe63a3a4b19d878; no voters:
I20260925 05:19:03.625149 1102 raft_consensus.cc:2781] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:03.625303 1102 raft_consensus.cc:686] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 1 LEADER]: Becoming Leader. State: Replica: d61acd1f9856438fbbe63a3a4b19d878, State: Running, Role: LEADER
I20260925 05:19:03.625403 1102 consensus_queue.cc:237] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } }
I20260925 05:19:03.626312 614 catalog_manager.cc:5495] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 reported cstate change: term changed from 0 to 1, leader changed from <none> to d61acd1f9856438fbbe63a3a4b19d878 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "d61acd1f9856438fbbe63a3a4b19d878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d61acd1f9856438fbbe63a3a4b19d878" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 41873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 } health_report { overall_health: UNKNOWN } } }
I20260925 05:19:03.634245 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.631856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.13.160.131:41873)
0925 05:19:03.631901 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.631903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.631912 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:03.631969 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.631969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.631993 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.632072 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:03.632232 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.632693 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.633445 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.633637 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.633710 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.633895 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:03.633990 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:03.634077 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.634118 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.634138 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:03.636011 761 raft_consensus.cc:1261] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 1 FOLLOWER]: Refusing update from remote peer d61acd1f9856438fbbe63a3a4b19d878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:03.636018 896 raft_consensus.cc:1261] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 1 FOLLOWER]: Refusing update from remote peer d61acd1f9856438fbbe63a3a4b19d878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:03.636376 1106 consensus_queue.cc:1035] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef3839360e24db9a104806f24abeb37" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 39247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:03.636538 1102 consensus_queue.cc:1035] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9d3d4f31ee44c7085a8b72f3a47e2d3" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 38353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:03.638747 1109 mvcc.cc:205] Tried to move back new op lower bound from 7333124274730627072 to 7333124274690416640. Current Snapshot: MvccSnapshot[applied={T|T < 7333124274730627072}]
I20260925 05:19:03.639129 1110 mvcc.cc:205] Tried to move back new op lower bound from 7333124274730627072 to 7333124274690416640. Current Snapshot: MvccSnapshot[applied={T|T < 7333124274730627072}]
I20260925 05:19:03.641258 614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:03.641355 614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:03.642113 614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f4dd532dd514d81a98c4b66e2352ac3: Sending DeleteTablet for 3 replicas of tablet 850a48e4b88b4aefa72acc682a841540
I20260925 05:19:03.642455 741 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a48e4b88b4aefa72acc682a841540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:03 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:48500
I20260925 05:19:03.642465 876 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a48e4b88b4aefa72acc682a841540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:03 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:41082
I20260925 05:19:03.642681 1119 tablet_replica.cc:330] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37: stopping tablet replica
I20260925 05:19:03.642688 1012 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a48e4b88b4aefa72acc682a841540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:03 UTC) from {username='kudu', principal='kudu/127.13.160.190@KRBTEST.COM'} at 127.0.0.1:52436
I20260925 05:19:03.642787 1119 raft_consensus.cc:2229] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:19:03.642933 1119 raft_consensus.cc:2258] T 850a48e4b88b4aefa72acc682a841540 P 3ef3839360e24db9a104806f24abeb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:03.643133 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 678
I20260925 05:19:03.647042 1121 tablet_replica.cc:330] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: stopping tablet replica
I20260925 05:19:03.647228 1121 raft_consensus.cc:2229] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:19:03.647418 1121 raft_consensus.cc:2258] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:03.647531 1120 tablet_replica.cc:330] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: stopping tablet replica
I20260925 05:19:03.647661 1120 raft_consensus.cc:2229] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:19:03.647778 1120 raft_consensus.cc:2258] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:03.648272 1120 ts_tablet_manager.cc:1905] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:03.649104 1121 ts_tablet_manager.cc:1905] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 05:19:03.650130 599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:39247 (error 108)
I20260925 05:19:03.650310 1120 ts_tablet_manager.cc:1918] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:03.650393 1120 log.cc:1196] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/850a48e4b88b4aefa72acc682a841540
I20260925 05:19:03.650677 1120 ts_tablet_manager.cc:1939] T 850a48e4b88b4aefa72acc682a841540 P a9d3d4f31ee44c7085a8b72f3a47e2d3: Deleting consensus metadata
I20260925 05:19:03.650790 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 813
I20260925 05:19:03.651877 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.651735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:39247 (local address 127.0.0.1:48504)
0925 05:19:03.651794 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.651837 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:39247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 05:19:03.652060 599 catalog_manager.cc:4584] TS 3ef3839360e24db9a104806f24abeb37 (127.13.160.129:39247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 850a48e4b88b4aefa72acc682a841540: Network error: Client connection negotiation failed: client connection to 127.13.160.129:39247: connect: Connection refused (error 111)
I20260925 05:19:03.655630 1121 ts_tablet_manager.cc:1918] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:03.655732 1121 log.cc:1196] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/850a48e4b88b4aefa72acc682a841540
I20260925 05:19:03.656013 1121 ts_tablet_manager.cc:1939] T 850a48e4b88b4aefa72acc682a841540 P d61acd1f9856438fbbe63a3a4b19d878: Deleting consensus metadata
I20260925 05:19:03.656551 602 catalog_manager.cc:4841] TS d61acd1f9856438fbbe63a3a4b19d878 (127.13.160.131:41873): tablet 850a48e4b88b4aefa72acc682a841540 (table test-table [id=fd569c01685246cba6a631de1e84d4b9]) successfully deleted
I20260925 05:19:03.657437 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.657319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:38353 (local address 127.0.0.1:41094)
0925 05:19:03.657371 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.657400 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.130:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 05:19:03.657457 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 948
I20260925 05:19:03.662586 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 583
2026-09-25T05:19:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1913 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T05:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:03Z Disabled control of system clock
I20260925 05:19:03.690474 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:39061
--webserver_interface=127.13.160.190
--webserver_port=0
--builtin_ntp_servers=127.13.160.148:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:39061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:19:03.773635 1126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:03.773897 1126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:03.773921 1126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:03.775424 1126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:03.775475 1126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:03.775487 1126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:03.775501 1126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:03.775513 1126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:03.777094 1126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:39061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:39061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:03.777329 1126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:03.777606 1126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:19:03.782667 1126 server_base.cc:1034] running on GCE node
W20260925 05:19:03.782627 1132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.782632 1134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.782627 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:03.783154 1126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:03.783522 1126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:03.784689 1126 hybrid_clock.cc:648] HybridClock initialized: now 1790313543784678 us; error 52 us; skew 500 ppm
I20260925 05:19:03.786006 1126 webserver.cc:460] Webserver started at http://127.13.160.190:46825/ using document root <none> and password file <none>
I20260925 05:19:03.786320 1126 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:03.786372 1126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:03.786556 1126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:03.787810 1126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "80eb30405f7d4858acf7d610e01d1660"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "33f8a3bb3885d8bf6d6d90c534348f4f"
server_key_iv: "d589f3d1eb1d4bd444e2d74e515d3f2d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.788208 1126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "80eb30405f7d4858acf7d610e01d1660"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "33f8a3bb3885d8bf6d6d90c534348f4f"
server_key_iv: "d589f3d1eb1d4bd444e2d74e515d3f2d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.789583 1126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:19:03.790369 1140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.790555 1126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:19:03.790678 1126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "80eb30405f7d4858acf7d610e01d1660"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "33f8a3bb3885d8bf6d6d90c534348f4f"
server_key_iv: "d589f3d1eb1d4bd444e2d74e515d3f2d"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.790761 1126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.810575 1126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.810931 1126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.811043 1126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.814849 1126 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:39061
I20260925 05:19:03.814980 1192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:39061 every 8 connection(s)
I20260925 05:19:03.815348 1126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:03.816350 1193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:03.818754 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: Bootstrap starting.
I20260925 05:19:03.819413 1193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:03.819739 1193 log.cc:824] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:03.820590 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: No bootstrap required, opened a new log
I20260925 05:19:03.822247 1193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } }
I20260925 05:19:03.822423 1193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:03.822470 1193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80eb30405f7d4858acf7d610e01d1660, State: Initialized, Role: FOLLOWER
I20260925 05:19:03.822662 1193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } }
I20260925 05:19:03.822752 1193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:03.822793 1193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:03.822845 1193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:03.823462 1193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } }
I20260925 05:19:03.823590 1193 leader_election.cc:302] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80eb30405f7d4858acf7d610e01d1660; no voters:
I20260925 05:19:03.823800 1193 leader_election.cc:288] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:03.823858 1196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:03.824072 1193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:03.824115 1196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [term 1 LEADER]: Becoming Leader. State: Replica: 80eb30405f7d4858acf7d610e01d1660, State: Running, Role: LEADER
I20260925 05:19:03.824312 1196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } }
I20260925 05:19:03.824715 1197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80eb30405f7d4858acf7d610e01d1660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } } }
I20260925 05:19:03.824820 1197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:03.825253 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1126
I20260925 05:19:03.825358 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:03.825419 1206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:03.825588 13954 external_mini_cluster.cc:1428] Setting key 19d2899112aff2954747baef1e1ea565
I20260925 05:19:03.824807 1198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80eb30405f7d4858acf7d610e01d1660. Latest consensus state: current_term: 1 leader_uuid: "80eb30405f7d4858acf7d610e01d1660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80eb30405f7d4858acf7d610e01d1660" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 39061 } } }
I20260925 05:19:03.825816 1198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:03.826152 1206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:03.827950 1206 catalog_manager.cc:1347] Generated new cluster ID: 92f9a92524eb4a7ca594c62a6143a802
I20260925 05:19:03.828011 1206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:03.831283 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.826134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.13.160.190:39061)
0925 05:19:03.826357 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.826365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.826391 (+ 26us) server_negotiation.cc:407] Connection header received
0925 05:19:03.826495 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.826501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.826567 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.826683 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.826966 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.828259 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.829175 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.829621 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.829760 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.830478 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.830482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.830482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:03.830679 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.830713 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:03.830724 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.830800 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.830876 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.831099 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260925 05:19:03.838841 1206 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:03.839473 1206 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:03.845108 1206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: Generated new TSK 0
I20260925 05:19:03.845371 1206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:03.847146 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.129:0
--local_ip_for_outbound_sockets=127.13.160.129
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--builtin_ntp_servers=127.13.160.148:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:19:03.935889 1218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:03.936159 1218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:03.936210 1218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:03.937510 1218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:03.937582 1218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:03.937628 1218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.129
I20260925 05:19:03.939179 1218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.160.129
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.129
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:03.939471 1218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:03.939802 1218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:03.945869 1223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.946004 1226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:03.945869 1224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:03.946319 1218 server_base.cc:1034] running on GCE node
I20260925 05:19:03.946545 1218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:03.946830 1218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:03.948009 1218 hybrid_clock.cc:648] HybridClock initialized: now 1790313543947992 us; error 33 us; skew 500 ppm
I20260925 05:19:03.949409 1218 webserver.cc:460] Webserver started at http://127.13.160.129:35817/ using document root <none> and password file <none>
I20260925 05:19:03.949685 1218 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:03.949760 1218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:03.949921 1218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:03.951156 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data/instance:
uuid: "d1a30114990843789ca96b073bd1ef5d"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "a7c9c776362f9a6e38ec30ab12293f9b"
server_key_iv: "21e3bfd700e4ec2122adda18be11f9e8"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.951541 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance:
uuid: "d1a30114990843789ca96b073bd1ef5d"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "a7c9c776362f9a6e38ec30ab12293f9b"
server_key_iv: "21e3bfd700e4ec2122adda18be11f9e8"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.953011 1218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:03.953836 1232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.953989 1218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:03.954102 1218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal
uuid: "d1a30114990843789ca96b073bd1ef5d"
format_stamp: "Formatted at 2026-09-25 05:19:03 on dist-test-slave-9tgz"
server_key: "a7c9c776362f9a6e38ec30ab12293f9b"
server_key_iv: "21e3bfd700e4ec2122adda18be11f9e8"
server_key_version: "encryptionkey@0"
I20260925 05:19:03.954196 1218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:03.964182 1218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:03.964545 1218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:03.964706 1218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:03.964953 1218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:03.965320 1218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:03.965354 1218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.965378 1218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:03.965425 1218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:03.971793 1218 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.129:44805
I20260925 05:19:03.971834 1346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.129:44805 every 8 connection(s)
I20260925 05:19:03.972249 1218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/data/info.pb
I20260925 05:19:03.972400 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1218
I20260925 05:19:03.972524 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-0/wal/instance
I20260925 05:19:03.972770 13954 external_mini_cluster.cc:1428] Setting key 8de3ed5c1c05b04412c61a81380315b1
I20260925 05:19:03.974134 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.130:0
--local_ip_for_outbound_sockets=127.13.160.130
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--builtin_ntp_servers=127.13.160.148:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 05:19:03.979066 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.973177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.129:34867 (local address 127.13.160.190:39061)
0925 05:19:03.973485 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:03.973487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:03.973753 (+ 266us) server_negotiation.cc:407] Connection header received
0925 05:19:03.974581 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:03.974582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:03.974596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:03.974637 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:03.975710 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.976343 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.977394 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.977613 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.977709 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.978713 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:03.978715 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:03.978715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:03.978721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:03.978737 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:03.978742 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:03.978823 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:03.978935 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 05:19:03.978975 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260925 05:19:03.979370 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:03.973245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:39061 (local address 127.13.160.129:34867)
0925 05:19:03.973608 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:03.973659 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:03.974446 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:03.974732 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:03.974737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:03.974749 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:03.975567 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.975576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.976446 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.976447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.977278 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:03.977279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:03.977722 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:03.977723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:03.977886 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:03.978533 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:03.978534 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:03.978572 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:03.978598 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:03.978828 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:03.978833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:03.978837 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:03.978942 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 05:19:03.979061 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260925 05:19:03.979936 1347 heartbeater.cc:347] Connected to a master server at 127.13.160.190:39061
I20260925 05:19:03.980072 1347 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:03.980324 1347 heartbeater.cc:511] Master 127.13.160.190:39061 requested a full tablet report, sending...
I20260925 05:19:03.980894 1157 ts_manager.cc:194] Registered new tserver with Master: d1a30114990843789ca96b073bd1ef5d (127.13.160.129:44805)
I20260925 05:19:03.981544 1157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.129:34867
W20260925 05:19:04.063362 1350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:04.063601 1350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:04.063627 1350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:04.064929 1350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:04.064989 1350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:04.065019 1350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.130
I20260925 05:19:04.066591 1350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.160.130
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.130
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:04.066929 1350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:04.067318 1350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:04.072546 1356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:04.072548 1358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:04.072743 1355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:04.072650 1350 server_base.cc:1034] running on GCE node
I20260925 05:19:04.073002 1350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:04.073290 1350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:04.074470 1350 hybrid_clock.cc:648] HybridClock initialized: now 1790313544074446 us; error 35 us; skew 500 ppm
I20260925 05:19:04.076056 1350 webserver.cc:460] Webserver started at http://127.13.160.130:42129/ using document root <none> and password file <none>
I20260925 05:19:04.076373 1350 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:04.076458 1350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:04.076607 1350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:04.077777 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data/instance:
uuid: "d573527355fd4f3ab4c38c968bb4e523"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "9dad26f43fe70628262311dd4d4e386a"
server_key_iv: "65a5a10e2979a4106fee35c4fbb2685e"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.078194 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance:
uuid: "d573527355fd4f3ab4c38c968bb4e523"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "9dad26f43fe70628262311dd4d4e386a"
server_key_iv: "65a5a10e2979a4106fee35c4fbb2685e"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.079698 1350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 05:19:04.080544 1364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.080737 1350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:04.080848 1350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal
uuid: "d573527355fd4f3ab4c38c968bb4e523"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "9dad26f43fe70628262311dd4d4e386a"
server_key_iv: "65a5a10e2979a4106fee35c4fbb2685e"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.080937 1350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:04.093495 1350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:04.093853 1350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:04.094044 1350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:04.094355 1350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:04.094717 1350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:04.094774 1350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.094815 1350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:04.094841 1350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.101109 1350 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.130:45347
I20260925 05:19:04.101151 1477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.130:45347 every 8 connection(s)
I20260925 05:19:04.101526 1350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/data/info.pb
I20260925 05:19:04.107235 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.102343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.130:44357 (local address 127.13.160.190:39061)
0925 05:19:04.102405 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.102406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.102636 (+ 230us) server_negotiation.cc:407] Connection header received
0925 05:19:04.103236 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.103238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.103248 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.103313 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:04.104026 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.104539 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.105652 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.105858 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.105922 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.106873 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:04.106874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:04.106874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:04.106880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:04.106923 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:04.106927 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:04.107009 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.107132 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.107166 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:19:04.107497 1480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.102293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:39061 (local address 127.13.160.130:44357)
0925 05:19:04.102491 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.102535 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.103129 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.103375 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.103378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.103388 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:04.103915 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.103919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.104666 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.104668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.105546 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.105548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.105947 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.105948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.106083 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.106714 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:04.106715 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:04.106749 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:04.106775 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:04.107018 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:04.107026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:04.107029 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.107129 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.107230 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260925 05:19:04.108085 1478 heartbeater.cc:347] Connected to a master server at 127.13.160.190:39061
I20260925 05:19:04.108203 1478 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:04.108397 1478 heartbeater.cc:511] Master 127.13.160.190:39061 requested a full tablet report, sending...
I20260925 05:19:04.108914 1157 ts_manager.cc:194] Registered new tserver with Master: d573527355fd4f3ab4c38c968bb4e523 (127.13.160.130:45347)
I20260925 05:19:04.109368 1157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.130:44357
I20260925 05:19:04.109964 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1350
I20260925 05:19:04.110049 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal/instance
I20260925 05:19:04.110271 13954 external_mini_cluster.cc:1428] Setting key b7870cde15cd2c020c093bf767641240
I20260925 05:19:04.111261 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.160.131:0
--local_ip_for_outbound_sockets=127.13.160.131
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--builtin_ntp_servers=127.13.160.148:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 05:19:04.194959 1481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:04.195233 1481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:04.195272 1481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:04.196555 1481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:04.196638 1481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:04.196686 1481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.160.131
I20260925 05:19:04.198192 1481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.160.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.160.131
--webserver_port=0
--tserver_master_addrs=127.13.160.190:39061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.160.131
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:04.198504 1481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:04.198848 1481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:19:04.204288 1481 server_base.cc:1034] running on GCE node
W20260925 05:19:04.204329 1489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:04.204448 1487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:04.204293 1486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:04.204813 1481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:04.205094 1481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:04.206285 1481 hybrid_clock.cc:648] HybridClock initialized: now 1790313544206269 us; error 38 us; skew 500 ppm
I20260925 05:19:04.207831 1481 webserver.cc:460] Webserver started at http://127.13.160.131:34931/ using document root <none> and password file <none>
I20260925 05:19:04.208122 1481 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:04.208210 1481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:04.208374 1481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:04.209532 1481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data/instance:
uuid: "d244c272327f45a0a1835555f2b33f44"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "145ad10012810c77b11256f32230ed0b"
server_key_iv: "4e18552c28ed86010b969ab4160b2280"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.209933 1481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance:
uuid: "d244c272327f45a0a1835555f2b33f44"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "145ad10012810c77b11256f32230ed0b"
server_key_iv: "4e18552c28ed86010b969ab4160b2280"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.211458 1481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:04.212332 1495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.212558 1481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:04.212675 1481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal
uuid: "d244c272327f45a0a1835555f2b33f44"
format_stamp: "Formatted at 2026-09-25 05:19:04 on dist-test-slave-9tgz"
server_key: "145ad10012810c77b11256f32230ed0b"
server_key_iv: "4e18552c28ed86010b969ab4160b2280"
server_key_version: "encryptionkey@0"
I20260925 05:19:04.212759 1481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:04.225554 1481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:04.225926 1481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:04.226091 1481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:04.226352 1481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 05:19:04.226716 1481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 05:19:04.226775 1481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.226821 1481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 05:19:04.226846 1481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:04.233098 1481 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.131:39257
I20260925 05:19:04.233139 1609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.131:39257 every 8 connection(s)
I20260925 05:19:04.233517 1481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/data/info.pb
I20260925 05:19:04.236423 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1481
I20260925 05:19:04.236549 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal/instance
I20260925 05:19:04.236795 13954 external_mini_cluster.cc:1428] Setting key 3e70fb2a38ab265d9b387cd9081ac721
I20260925 05:19:04.239586 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.234319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:45639 (local address 127.13.160.190:39061)
0925 05:19:04.234397 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.234399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.234687 (+ 288us) server_negotiation.cc:407] Connection header received
0925 05:19:04.235319 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.235320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.235331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.235393 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:04.236072 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.236621 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.238085 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.238294 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.238352 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.239315 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:04.239317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:04.239317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:04.239322 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:04.239338 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:04.239341 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:04.239413 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.239483 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.239515 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:19:04.239814 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.234302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:39061 (local address 127.13.160.131:45639)
0925 05:19:04.234554 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.234596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.235223 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.235455 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.235458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.235468 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:04.235989 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.235993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.236738 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.236740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.238010 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.238013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.238402 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.238404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.238553 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.239170 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 05:19:04.239171 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:04.239204 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 05:19:04.239230 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:04.239416 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:04.239419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:04.239422 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.239471 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.239561 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":123,"threads_started":1}
I20260925 05:19:04.240305 1610 heartbeater.cc:347] Connected to a master server at 127.13.160.190:39061
I20260925 05:19:04.240396 1610 heartbeater.cc:464] Registering TS with master...
I20260925 05:19:04.240553 1610 heartbeater.cc:511] Master 127.13.160.190:39061 requested a full tablet report, sending...
I20260925 05:19:04.240888 1157 ts_manager.cc:194] Registered new tserver with Master: d244c272327f45a0a1835555f2b33f44 (127.13.160.131:39257)
I20260925 05:19:04.241285 1157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.160.131:45639
I20260925 05:19:04.248796 13954 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 05:19:04.253635 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.250397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.13.160.190:39061)
0925 05:19:04.250481 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.250483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.250587 (+ 104us) server_negotiation.cc:407] Connection header received
0925 05:19:04.250672 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.250673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.250696 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.250755 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:04.250995 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.251511 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.252344 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.252546 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.252628 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.253338 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:04.253339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:04.253339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:04.253345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:04.253365 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:04.253374 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:04.253436 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.253498 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.253534 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 05:19:04.255581 1157 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 05:19:04.257350 1157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 05:19:04.264391 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.260031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.13.160.130:45347)
0925 05:19:04.260249 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.260254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.260285 (+ 31us) server_negotiation.cc:407] Connection header received
0925 05:19:04.260336 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.260338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.260357 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.260440 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:04.260707 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.261515 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.262996 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.263800 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263878 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.263984 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.264201 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.264277 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260925 05:19:04.264376 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.259842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44805 (local address 127.0.0.1:34428)
0925 05:19:04.260034 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.260083 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.260160 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.260658 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.260659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.260660 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:04.260696 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.260698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.261991 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.261992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.263135 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.263137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.264101 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.264102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.264180 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.264184 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.264235 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.264282 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":51,"thread_start_us":66,"threads_started":1}
I20260925 05:19:04.264416 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.260313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.13.160.131:39257)
0925 05:19:04.260691 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.260695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.260706 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:04.260744 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.260746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.260761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.260839 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:04.260928 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.262005 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263243 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.263743 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263872 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.263984 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.264212 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.264333 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":83,"threads_started":1}
I20260925 05:19:04.264802 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.259985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.13.160.129:44805)
0925 05:19:04.260493 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.260497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.260508 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:04.260547 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.260549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.260565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.260645 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:04.260757 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.261894 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263225 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.263931 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263995 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.264066 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.264689 (+ 623us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.264727 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":100,"threads_started":1}
I20260925 05:19:04.264402 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.260061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.131:39257 (local address 127.0.0.1:59668)
0925 05:19:04.260332 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.260341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.260378 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.260834 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.260834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.260836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:04.260863 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.260864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.262097 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.262098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.263100 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.263103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.264014 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.264015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.264067 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.264078 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.264153 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.264331 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":123,"thread_start_us":69,"threads_started":1}
I20260925 05:19:04.264969 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.259842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:45347 (local address 127.0.0.1:59422)
0925 05:19:04.260184 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.260193 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.260236 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.260530 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.260533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.260535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:04.260593 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.260595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.261611 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.261612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.262865 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.262868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.263918 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.263919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.264056 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.264078 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.264159 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.264230 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":2,"spinlock_wait_cycles":896,"thread_start_us":114,"threads_started":1}
I20260925 05:19:04.265748 1280 tablet_service.cc:1467] Processing CreateTablet for tablet 6880dd539f45406cbc9771d9975347fa (DEFAULT_TABLE table=test-table [id=40775211476544d98bc184af612c2316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:04.265753 1412 tablet_service.cc:1467] Processing CreateTablet for tablet 6880dd539f45406cbc9771d9975347fa (DEFAULT_TABLE table=test-table [id=40775211476544d98bc184af612c2316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:04.266139 1280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880dd539f45406cbc9771d9975347fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:04.266144 1412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880dd539f45406cbc9771d9975347fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:04.269200 1625 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Bootstrap starting.
I20260925 05:19:04.269276 1626 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Bootstrap starting.
I20260925 05:19:04.269977 1626 tablet_bootstrap.cc:654] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:04.270033 1625 tablet_bootstrap.cc:654] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:04.270143 1544 tablet_service.cc:1467] Processing CreateTablet for tablet 6880dd539f45406cbc9771d9975347fa (DEFAULT_TABLE table=test-table [id=40775211476544d98bc184af612c2316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 05:19:04.270313 1626 log.cc:824] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:04.270356 1625 log.cc:824] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:04.270541 1544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6880dd539f45406cbc9771d9975347fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:04.271194 1626 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: No bootstrap required, opened a new log
I20260925 05:19:04.271291 1626 ts_tablet_manager.cc:1397] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 05:19:04.271687 1625 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: No bootstrap required, opened a new log
I20260925 05:19:04.271760 1625 ts_tablet_manager.cc:1397] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 05:19:04.273280 1626 raft_consensus.cc:348] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.273449 1626 raft_consensus.cc:374] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:04.273494 1626 raft_consensus.cc:729] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1a30114990843789ca96b073bd1ef5d, State: Initialized, Role: FOLLOWER
I20260925 05:19:04.273608 1626 consensus_queue.cc:260] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.273926 1626 ts_tablet_manager.cc:1428] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:04.273947 1347 heartbeater.cc:503] Master 127.13.160.190:39061 was elected leader, sending a full tablet report...
I20260925 05:19:04.274300 1630 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Bootstrap starting.
I20260925 05:19:04.274389 1625 raft_consensus.cc:348] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.274556 1625 raft_consensus.cc:374] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:04.274595 1625 raft_consensus.cc:729] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d573527355fd4f3ab4c38c968bb4e523, State: Initialized, Role: FOLLOWER
I20260925 05:19:04.274708 1625 consensus_queue.cc:260] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.274996 1625 ts_tablet_manager.cc:1428] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 05:19:04.275024 1478 heartbeater.cc:503] Master 127.13.160.190:39061 was elected leader, sending a full tablet report...
I20260925 05:19:04.275035 1630 tablet_bootstrap.cc:654] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:04.275393 1630 log.cc:824] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:04.276324 1630 tablet_bootstrap.cc:492] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: No bootstrap required, opened a new log
I20260925 05:19:04.276396 1630 ts_tablet_manager.cc:1397] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 05:19:04.278365 1630 raft_consensus.cc:348] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.278503 1630 raft_consensus.cc:374] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:04.278527 1630 raft_consensus.cc:729] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d244c272327f45a0a1835555f2b33f44, State: Initialized, Role: FOLLOWER
I20260925 05:19:04.278683 1630 consensus_queue.cc:260] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.278988 1630 ts_tablet_manager.cc:1428] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 05:19:04.279060 1610 heartbeater.cc:503] Master 127.13.160.190:39061 was elected leader, sending a full tablet report...
W20260925 05:19:04.352124 1479 tablet.cc:2369] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:19:04.433995 1633 raft_consensus.cc:482] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 05:19:04.434177 1633 raft_consensus.cc:504] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.434610 1633 leader_election.cc:288] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d573527355fd4f3ab4c38c968bb4e523 (127.13.160.130:45347), d1a30114990843789ca96b073bd1ef5d (127.13.160.129:44805)
I20260925 05:19:04.437525 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.434880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:55439 (local address 127.13.160.129:44805)
0925 05:19:04.434947 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.434949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.434960 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:04.435059 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.435060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.435072 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.435179 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:04.435335 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.435810 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.436786 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.437154 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.437248 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.437257 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.437404 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.437447 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 05:19:04.437525 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.434821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44805 (local address 127.13.160.131:55439)
0925 05:19:04.434884 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.434897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.434961 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.435160 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.435162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.435164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:04.435252 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.435254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.435904 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.435905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.436679 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.436682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.437247 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.437248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.437328 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.437331 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.437398 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.437446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 05:19:04.437611 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.434929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.160.131:53569 (local address 127.13.160.130:45347)
0925 05:19:04.434977 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.434979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.435117 (+ 138us) server_negotiation.cc:407] Connection header received
0925 05:19:04.435218 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.435229 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.435241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.435318 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 05:19:04.435478 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.435940 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.436874 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.437242 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.437329 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.437337 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.437492 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.437536 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 05:19:04.437664 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.434839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.130:45347 (local address 127.13.160.131:53569)
0925 05:19:04.435032 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.435043 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:04.435137 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:04.435322 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:04.435323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:04.435325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 05:19:04.435397 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.435398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.436034 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.436035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.436779 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:04.436781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.437329 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.437330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:04.437408 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.437411 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 05:19:04.437473 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 05:19:04.437514 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":124,"threads_started":1}
I20260925 05:19:04.437958 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6880dd539f45406cbc9771d9975347fa" candidate_uuid: "d244c272327f45a0a1835555f2b33f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a30114990843789ca96b073bd1ef5d" is_pre_election: true
I20260925 05:19:04.438063 1432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6880dd539f45406cbc9771d9975347fa" candidate_uuid: "d244c272327f45a0a1835555f2b33f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d573527355fd4f3ab4c38c968bb4e523" is_pre_election: true
I20260925 05:19:04.438158 1300 raft_consensus.cc:2454] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d244c272327f45a0a1835555f2b33f44 in term 0.
I20260925 05:19:04.438246 1432 raft_consensus.cc:2454] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d244c272327f45a0a1835555f2b33f44 in term 0.
I20260925 05:19:04.438373 1497 leader_election.cc:302] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1a30114990843789ca96b073bd1ef5d, d244c272327f45a0a1835555f2b33f44; no voters:
I20260925 05:19:04.438545 1633 raft_consensus.cc:2781] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 05:19:04.438620 1633 raft_consensus.cc:482] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 05:19:04.438639 1633 raft_consensus.cc:3037] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:04.439466 1633 raft_consensus.cc:504] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.439628 1633 leader_election.cc:288] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [CANDIDATE]: Term 1 election: Requested vote from peers d573527355fd4f3ab4c38c968bb4e523 (127.13.160.130:45347), d1a30114990843789ca96b073bd1ef5d (127.13.160.129:44805)
I20260925 05:19:04.439833 1432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6880dd539f45406cbc9771d9975347fa" candidate_uuid: "d244c272327f45a0a1835555f2b33f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d573527355fd4f3ab4c38c968bb4e523"
I20260925 05:19:04.439851 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6880dd539f45406cbc9771d9975347fa" candidate_uuid: "d244c272327f45a0a1835555f2b33f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a30114990843789ca96b073bd1ef5d"
I20260925 05:19:04.439925 1300 raft_consensus.cc:3037] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:04.439925 1432 raft_consensus.cc:3037] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:04.440882 1432 raft_consensus.cc:2454] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d244c272327f45a0a1835555f2b33f44 in term 1.
I20260925 05:19:04.440882 1300 raft_consensus.cc:2454] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d244c272327f45a0a1835555f2b33f44 in term 1.
I20260925 05:19:04.441059 1496 leader_election.cc:302] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d244c272327f45a0a1835555f2b33f44, d573527355fd4f3ab4c38c968bb4e523; no voters:
I20260925 05:19:04.441184 1633 raft_consensus.cc:2781] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:04.441349 1633 raft_consensus.cc:686] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 1 LEADER]: Becoming Leader. State: Replica: d244c272327f45a0a1835555f2b33f44, State: Running, Role: LEADER
I20260925 05:19:04.441462 1633 consensus_queue.cc:237] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } }
I20260925 05:19:04.442435 1156 catalog_manager.cc:5495] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 reported cstate change: term changed from 0 to 1, leader changed from <none> to d244c272327f45a0a1835555f2b33f44 (127.13.160.131). New cstate: current_term: 1 leader_uuid: "d244c272327f45a0a1835555f2b33f44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d244c272327f45a0a1835555f2b33f44" member_type: VOTER last_known_addr { host: "127.13.160.131" port: 39257 } health_report { overall_health: HEALTHY } } }
W20260925 05:19:04.473205 1348 tablet.cc:2369] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 05:19:04.484090 1611 tablet.cc:2369] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 05:19:04.490480 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.487951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.13.160.131:39257)
0925 05:19:04.488017 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:04.488020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:04.488031 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:04.488085 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:04.488086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:04.488105 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:04.488196 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:04.488345 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.488843 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.489630 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:04.489834 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:04.489914 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:04.490105 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:04.490222 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:04.490317 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:04.490340 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 05:19:04.490363 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 05:19:04.492517 1300 raft_consensus.cc:1261] T 6880dd539f45406cbc9771d9975347fa P d1a30114990843789ca96b073bd1ef5d [term 1 FOLLOWER]: Refusing update from remote peer d244c272327f45a0a1835555f2b33f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:04.492516 1432 raft_consensus.cc:1261] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 1 FOLLOWER]: Refusing update from remote peer d244c272327f45a0a1835555f2b33f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:04.492926 1633 consensus_queue.cc:1035] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1a30114990843789ca96b073bd1ef5d" member_type: VOTER last_known_addr { host: "127.13.160.129" port: 44805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:04.493112 1635 consensus_queue.cc:1035] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d573527355fd4f3ab4c38c968bb4e523" member_type: VOTER last_known_addr { host: "127.13.160.130" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:04.495772 1640 mvcc.cc:205] Tried to move back new op lower bound from 7333124278238253056 to 7333124278033088512. Current Snapshot: MvccSnapshot[applied={T|T < 7333124278238253056}]
I20260925 05:19:04.496006 1639 mvcc.cc:205] Tried to move back new op lower bound from 7333124278238253056 to 7333124278033088512. Current Snapshot: MvccSnapshot[applied={T|T < 7333124278238253056}]
I20260925 05:19:04.498006 1157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:04.498121 1157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 05:19:04.498971 1157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80eb30405f7d4858acf7d610e01d1660: Sending DeleteTablet for 3 replicas of tablet 6880dd539f45406cbc9771d9975347fa
I20260925 05:19:04.499346 1280 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880dd539f45406cbc9771d9975347fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:04 UTC) from {username='slave'} at 127.0.0.1:34428
I20260925 05:19:04.499447 1412 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880dd539f45406cbc9771d9975347fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:04 UTC) from {username='slave'} at 127.0.0.1:59422
I20260925 05:19:04.499468 1544 tablet_service.cc:1514] Processing DeleteTablet for tablet 6880dd539f45406cbc9771d9975347fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 05:19:04 UTC) from {username='slave'} at 127.0.0.1:59668
I20260925 05:19:04.499660 1649 tablet_replica.cc:330] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: stopping tablet replica
I20260925 05:19:04.499760 1649 raft_consensus.cc:2229] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 05:19:04.499785 1650 tablet_replica.cc:330] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: stopping tablet replica
I20260925 05:19:04.499936 1649 raft_consensus.cc:2258] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:04.499965 1650 raft_consensus.cc:2229] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 1 LEADER]: Raft consensus shutting down.
I20260925 05:19:04.500077 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1218
I20260925 05:19:04.500206 1650 raft_consensus.cc:2258] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 05:19:04.500443 1649 ts_tablet_manager.cc:1905] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:04.500730 1650 ts_tablet_manager.cc:1905] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 05:19:04.502369 1649 ts_tablet_manager.cc:1918] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:04.502453 1649 log.cc:1196] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-1/wal/wals/6880dd539f45406cbc9771d9975347fa
I20260925 05:19:04.502666 1650 ts_tablet_manager.cc:1918] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 05:19:04.502729 1649 ts_tablet_manager.cc:1939] T 6880dd539f45406cbc9771d9975347fa P d573527355fd4f3ab4c38c968bb4e523: Deleting consensus metadata
I20260925 05:19:04.502846 1650 log.cc:1196] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Deleting WAL directory at /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790313300911901-13954-0/minicluster-data/ts-2/wal/wals/6880dd539f45406cbc9771d9975347fa
I20260925 05:19:04.503207 1650 ts_tablet_manager.cc:1939] T 6880dd539f45406cbc9771d9975347fa P d244c272327f45a0a1835555f2b33f44: Deleting consensus metadata
I20260925 05:19:04.503219 1141 catalog_manager.cc:4841] TS d573527355fd4f3ab4c38c968bb4e523 (127.13.160.130:45347): tablet 6880dd539f45406cbc9771d9975347fa (table test-table [id=40775211476544d98bc184af612c2316]) successfully deleted
I20260925 05:19:04.503664 1141 catalog_manager.cc:4841] TS d244c272327f45a0a1835555f2b33f44 (127.13.160.131:39257): tablet 6880dd539f45406cbc9771d9975347fa (table test-table [id=40775211476544d98bc184af612c2316]) successfully deleted
W20260925 05:19:04.508448 1142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.160.129:44805 (error 108)
I20260925 05:19:04.508811 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1350
I20260925 05:19:04.508992 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:04.508859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.129:44805 (local address 127.0.0.1:34444)
0925 05:19:04.508914 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:04.508955 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.129:44805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 05:19:04.509480 1142 catalog_manager.cc:4584] TS d1a30114990843789ca96b073bd1ef5d (127.13.160.129:44805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6880dd539f45406cbc9771d9975347fa: Network error: Client connection negotiation failed: client connection to 127.13.160.129:44805: connect: Connection refused (error 111)
I20260925 05:19:04.514777 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1481
I20260925 05:19:04.519970 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1126
2026-09-25T05:19:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (855 ms)
[----------] 6 tests from EncryptionPolicyTest (8328 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:04 dist-test-slave-9tgz krb5kdc[1654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:04 dist-test-slave-9tgz krb5kdc[1654](info): set up 2 sockets
Sep 25 05:19:04 dist-test-slave-9tgz krb5kdc[1654](info): commencing operation
krb5kdc: starting...
W20260925 05:19:05.559792 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:05.680181 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:05.764008 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:05.764326 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:05.764374 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:05.765790 1667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:05.765885 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:05.765911 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:05.765933 1667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:05.765962 1667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:05.767731 1667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:05.768067 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:05.768338 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:05.773643 1675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:05.773689 1667 server_base.cc:1034] running on GCE node
W20260925 05:19:05.773653 1672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:05.773670 1673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:05.774214 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:05.774508 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:05.775687 1667 hybrid_clock.cc:648] HybridClock initialized: now 1790313545775673 us; error 49 us; skew 500 ppm
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313545, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:05.778396 1667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:05.778949 1667 webserver.cc:460] Webserver started at http://127.0.0.1:44295/ using document root <none> and password file <none>
I20260925 05:19:05.779219 1667 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:05.779294 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:05.779448 1667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:05.780571 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "7317d9cf22554fba83be64abdb520484"
format_stamp: "Formatted at 2026-09-25 05:19:05 on dist-test-slave-9tgz"
server_key: "1bc25d4012324cf1f1351c9611b26265"
server_key_iv: "6b939aae84a49e518f711e8000e0e33a"
server_key_version: "encryptionkey@0"
I20260925 05:19:05.781026 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "7317d9cf22554fba83be64abdb520484"
format_stamp: "Formatted at 2026-09-25 05:19:05 on dist-test-slave-9tgz"
server_key: "1bc25d4012324cf1f1351c9611b26265"
server_key_iv: "6b939aae84a49e518f711e8000e0e33a"
server_key_version: "encryptionkey@0"
I20260925 05:19:05.782439 1667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:05.783272 1682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:05.783473 1667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:19:05.783586 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "7317d9cf22554fba83be64abdb520484"
format_stamp: "Formatted at 2026-09-25 05:19:05 on dist-test-slave-9tgz"
server_key: "1bc25d4012324cf1f1351c9611b26265"
server_key_iv: "6b939aae84a49e518f711e8000e0e33a"
server_key_version: "encryptionkey@0"
I20260925 05:19:05.783670 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:05.815937 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:05.818564 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:05.818759 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:05.822706 1667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42701
I20260925 05:19:05.822781 1734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42701 every 8 connection(s)
I20260925 05:19:05.823220 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:05.824167 1735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:05.826102 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1667
I20260925 05:19:05.826244 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:05.826437 13954 external_mini_cluster.cc:1428] Setting key 31e8776a381866dbdb1f36bc3b98484f
I20260925 05:19:05.827168 1735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484: Bootstrap starting.
I20260925 05:19:05.827852 1735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:05.828163 1735 log.cc:824] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:05.828893 1735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484: No bootstrap required, opened a new log
I20260925 05:19:05.830757 1735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260925 05:19:05.830996 1735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:05.831051 1735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7317d9cf22554fba83be64abdb520484, State: Initialized, Role: FOLLOWER
I20260925 05:19:05.831250 1735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260925 05:19:05.831363 1735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:05.831403 1735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:05.831461 1735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:05.832355 1735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260925 05:19:05.832496 1735 leader_election.cc:302] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7317d9cf22554fba83be64abdb520484; no voters:
I20260925 05:19:05.832723 1735 leader_election.cc:288] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:05.832788 1740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:05.832983 1740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [term 1 LEADER]: Becoming Leader. State: Replica: 7317d9cf22554fba83be64abdb520484, State: Running, Role: LEADER
I20260925 05:19:05.833017 1735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:05.833084 1740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260925 05:19:05.833539 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7317d9cf22554fba83be64abdb520484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
I20260925 05:19:05.833527 1740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7317d9cf22554fba83be64abdb520484. Latest consensus state: current_term: 1 leader_uuid: "7317d9cf22554fba83be64abdb520484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7317d9cf22554fba83be64abdb520484" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 05:19:05.833633 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:05.833639 1740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:05.833847 1747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:05.834374 1747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:05.836498 1747 catalog_manager.cc:1347] Generated new cluster ID: de4ff2d560fa41a28c3001e025c09be5
I20260925 05:19:05.836555 1747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:05.838692 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:05.827040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43875 (local address 127.0.0.1:42701)
0925 05:19:05.827267 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:05.827278 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:05.827315 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:19:05.827780 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:05.827787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:05.827840 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:05.828019 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:05.828764 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:05.829730 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:05.830503 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:05.831011 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:05.831153 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:05.834296 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:05.834307 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:05.834308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:05.834321 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:05.836481 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:05.837024 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:05.837028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:05.837028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:05.837132 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:05.837385 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:05.837386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:05.837387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:05.837715 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:05.837864 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:05.838325 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 05:19:05.838434 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260925 05:19:05.839820 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:05.849264 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:05.841451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48885 (local address 127.0.0.1:42701)
0925 05:19:05.841486 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:05.841488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:05.841497 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:05.841544 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:05.841545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:05.841554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:05.841614 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:05.842267 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:05.843085 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:05.844101 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:05.844464 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:05.844544 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:05.846059 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:05.846061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:05.846061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:05.846069 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:05.847943 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:05.848371 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:05.848372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:05.848373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:05.848439 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:05.848675 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:05.848676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:05.848676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:05.848821 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:05.848930 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:05.849086 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 05:19:05.849157 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:19:05.862844 1747 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:05.863490 1747 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:05.866963 1747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7317d9cf22554fba83be64abdb520484: Generated new TSK 0
I20260925 05:19:05.867187 1747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:05.872562 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1667
W20260925 05:19:05.877256 1652 connection.cc:535] client connection to 127.0.0.1:42701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42701): Connection reset by peer (error 104)
2026-09-25T05:19:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1766](info): set up 2 sockets
Sep 25 05:19:05 dist-test-slave-9tgz krb5kdc[1766](info): commencing operation
krb5kdc: starting...
W20260925 05:19:06.904297 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:06 dist-test-slave-9tgz krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:07.026157 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:07.110793 1779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:07.111071 1779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:07.111097 1779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:07.112378 1779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:07.112434 1779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:07.112449 1779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:07.112462 1779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:07.112474 1779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:07.114084 1779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:07.114318 1779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:07.114533 1779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:07.119596 1785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:07.119616 1786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:07.119799 1779 server_base.cc:1034] running on GCE node
W20260925 05:19:07.119681 1788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:07.120122 1779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:07.120491 1779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:07.121660 1779 hybrid_clock.cc:648] HybridClock initialized: now 1790313547121656 us; error 38 us; skew 500 ppm
Sep 25 05:19:07 dist-test-slave-9tgz krb5kdc[1766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:07.124330 1779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:07.124800 1779 webserver.cc:460] Webserver started at http://127.0.0.1:33947/ using document root <none> and password file <none>
I20260925 05:19:07.125038 1779 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:07.125118 1779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:07.125272 1779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:07.126417 1779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "47ffcb47a1634d27b078264dc120f327"
format_stamp: "Formatted at 2026-09-25 05:19:07 on dist-test-slave-9tgz"
server_key: "8329e5fe77a57c17550ff4d3d563e902"
server_key_iv: "8ce4a8179017ef98df8746073ff7a794"
server_key_version: "encryptionkey@0"
I20260925 05:19:07.126796 1779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "47ffcb47a1634d27b078264dc120f327"
format_stamp: "Formatted at 2026-09-25 05:19:07 on dist-test-slave-9tgz"
server_key: "8329e5fe77a57c17550ff4d3d563e902"
server_key_iv: "8ce4a8179017ef98df8746073ff7a794"
server_key_version: "encryptionkey@0"
I20260925 05:19:07.128252 1779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:19:07.129081 1795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:07.129271 1779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:19:07.129386 1779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "47ffcb47a1634d27b078264dc120f327"
format_stamp: "Formatted at 2026-09-25 05:19:07 on dist-test-slave-9tgz"
server_key: "8329e5fe77a57c17550ff4d3d563e902"
server_key_iv: "8ce4a8179017ef98df8746073ff7a794"
server_key_version: "encryptionkey@0"
I20260925 05:19:07.129469 1779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:07.147895 1779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:07.149976 1779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:07.150171 1779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:07.154260 1779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36631
I20260925 05:19:07.154299 1847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36631 every 8 connection(s)
I20260925 05:19:07.154687 1779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:07.155722 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:07.158248 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327: Bootstrap starting.
I20260925 05:19:07.158986 1848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:07.159294 1848 log.cc:824] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:07.160053 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327: No bootstrap required, opened a new log
I20260925 05:19:07.161590 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1779
I20260925 05:19:07.161705 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:07.161919 13954 external_mini_cluster.cc:1428] Setting key a903cfd45d8f563d7f25def9ff49c328
I20260925 05:19:07.162096 1848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260925 05:19:07.162344 1848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:07.162413 1848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ffcb47a1634d27b078264dc120f327, State: Initialized, Role: FOLLOWER
I20260925 05:19:07.162614 1848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260925 05:19:07.162720 1848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:07.162755 1848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:07.162829 1848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:07.163589 1848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260925 05:19:07.163688 1848 leader_election.cc:302] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47ffcb47a1634d27b078264dc120f327; no voters:
I20260925 05:19:07.163934 1848 leader_election.cc:288] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:07.163954 1853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:07.164088 1853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [term 1 LEADER]: Becoming Leader. State: Replica: 47ffcb47a1634d27b078264dc120f327, State: Running, Role: LEADER
I20260925 05:19:07.164179 1853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260925 05:19:07.164245 1848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:07.164577 1855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47ffcb47a1634d27b078264dc120f327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } } }
I20260925 05:19:07.164664 1855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:07.164825 1854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47ffcb47a1634d27b078264dc120f327. Latest consensus state: current_term: 1 leader_uuid: "47ffcb47a1634d27b078264dc120f327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ffcb47a1634d27b078264dc120f327" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } } }
I20260925 05:19:07.164876 1854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:07.165256 1858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:07.165956 1858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:07.167696 1858 catalog_manager.cc:1347] Generated new cluster ID: 989b946fbf434495b062c6c19eb13242
I20260925 05:19:07.167754 1858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:07.169193 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:07.162599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38381 (local address 127.0.0.1:36631)
0925 05:19:07.163184 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:07.163193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:07.163226 (+ 33us) server_negotiation.cc:407] Connection header received
0925 05:19:07.163554 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:07.163561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:07.163611 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:07.163781 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:07.164118 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:07.165769 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:07.166951 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:07.167515 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:07.167656 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:07.168482 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:07.168486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:07.168486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:07.168689 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:07.168723 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:07.168736 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:07.168836 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:07.168922 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 05:19:07.168955 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":108,"threads_started":1}
I20260925 05:19:07.170223 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:07.175274 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:07.171758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38117 (local address 127.0.0.1:36631)
0925 05:19:07.171799 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:07.171801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:07.171914 (+ 113us) server_negotiation.cc:407] Connection header received
0925 05:19:07.171981 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:07.171982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:07.171994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:07.172047 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:07.172218 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:07.172999 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:07.173782 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:07.174055 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:07.174135 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:07.174944 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:07.174946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:07.174946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:07.174952 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:07.174968 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:07.174972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:07.175067 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:07.175136 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 05:19:07.175151 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:07.187001 1858 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:07.187618 1858 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:07.195700 1858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47ffcb47a1634d27b078264dc120f327: Generated new TSK 0
I20260925 05:19:07.195945 1858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:07.202616 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1779
W20260925 05:19:07.207130 1764 connection.cc:535] client connection to 127.0.0.1:36631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36631): Connection reset by peer (error 104)
2026-09-25T05:19:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:07 dist-test-slave-9tgz krb5kdc[1879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:07 dist-test-slave-9tgz krb5kdc[1879](info): set up 2 sockets
Sep 25 05:19:07 dist-test-slave-9tgz krb5kdc[1879](info): commencing operation
krb5kdc: starting...
W20260925 05:19:08.287940 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:08 dist-test-slave-9tgz krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:08.408185 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44957
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:08.491397 1892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:08.491619 1892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:08.491643 1892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:08.492769 1892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:08.492806 1892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:08.492820 1892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:08.492834 1892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:08.492846 1892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:08.494315 1892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:08.494531 1892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:08.494728 1892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:08.499682 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:08.499686 1898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:08.499682 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:08.499944 1892 server_base.cc:1034] running on GCE node
I20260925 05:19:08.500244 1892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:08.500572 1892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:08.501773 1892 hybrid_clock.cc:648] HybridClock initialized: now 1790313548501765 us; error 39 us; skew 500 ppm
Sep 25 05:19:08 dist-test-slave-9tgz krb5kdc[1879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313548, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:08.510331 1892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:08.510852 1892 webserver.cc:460] Webserver started at http://127.0.0.1:35265/ using document root <none> and password file <none>
I20260925 05:19:08.511157 1892 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:08.511237 1892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:08.511394 1892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:08.512526 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "e59a091b668746a3be09f462cb9f45a8"
format_stamp: "Formatted at 2026-09-25 05:19:08 on dist-test-slave-9tgz"
server_key: "e0241ac9e3d5ffd1507d7529c1f65c93"
server_key_iv: "0a51b560798cd513379ffdb91ae037c2"
server_key_version: "encryptionkey@0"
I20260925 05:19:08.512948 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "e59a091b668746a3be09f462cb9f45a8"
format_stamp: "Formatted at 2026-09-25 05:19:08 on dist-test-slave-9tgz"
server_key: "e0241ac9e3d5ffd1507d7529c1f65c93"
server_key_iv: "0a51b560798cd513379ffdb91ae037c2"
server_key_version: "encryptionkey@0"
I20260925 05:19:08.514402 1892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:19:08.515317 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:08.515520 1892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:19:08.515648 1892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "e59a091b668746a3be09f462cb9f45a8"
format_stamp: "Formatted at 2026-09-25 05:19:08 on dist-test-slave-9tgz"
server_key: "e0241ac9e3d5ffd1507d7529c1f65c93"
server_key_iv: "0a51b560798cd513379ffdb91ae037c2"
server_key_version: "encryptionkey@0"
I20260925 05:19:08.515731 1892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:08.519286 1892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:08.521360 1892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:08.521528 1892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:08.525627 1892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44957
I20260925 05:19:08.525658 1960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44957 every 8 connection(s)
I20260925 05:19:08.526036 1892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:08.527032 1961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:08.529289 1961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8: Bootstrap starting.
I20260925 05:19:08.529829 1961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:08.530071 1961 log.cc:824] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:08.530745 1961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8: No bootstrap required, opened a new log
I20260925 05:19:08.532428 1961 raft_consensus.cc:348] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260925 05:19:08.532577 1961 raft_consensus.cc:374] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:08.532598 1961 raft_consensus.cc:729] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59a091b668746a3be09f462cb9f45a8, State: Initialized, Role: FOLLOWER
I20260925 05:19:08.532745 1961 consensus_queue.cc:260] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260925 05:19:08.532829 1961 raft_consensus.cc:388] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:08.532853 1961 raft_consensus.cc:482] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:08.532881 1961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:08.533381 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 1892
I20260925 05:19:08.533397 1961 raft_consensus.cc:504] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260925 05:19:08.533501 1961 leader_election.cc:302] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e59a091b668746a3be09f462cb9f45a8; no voters:
I20260925 05:19:08.533540 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:08.533718 1961 leader_election.cc:288] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:08.533744 1964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:08.533744 13954 external_mini_cluster.cc:1428] Setting key ca0e30e3c9ffd5fb7a575f03ebdc76b9
I20260925 05:19:08.533864 1964 raft_consensus.cc:686] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [term 1 LEADER]: Becoming Leader. State: Replica: e59a091b668746a3be09f462cb9f45a8, State: Running, Role: LEADER
I20260925 05:19:08.533999 1964 consensus_queue.cc:237] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260925 05:19:08.534001 1961 sys_catalog.cc:564] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:08.535451 1965 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e59a091b668746a3be09f462cb9f45a8. Latest consensus state: current_term: 1 leader_uuid: "e59a091b668746a3be09f462cb9f45a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } } }
I20260925 05:19:08.535570 1965 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:08.535881 1979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:08.535846 1966 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e59a091b668746a3be09f462cb9f45a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59a091b668746a3be09f462cb9f45a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } } }
I20260925 05:19:08.536252 1966 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:08.536469 1968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:08.534308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45191 (local address 127.0.0.1:44957)
0925 05:19:08.534505 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:08.534515 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:08.534555 (+ 40us) server_negotiation.cc:407] Connection header received
0925 05:19:08.534709 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:08.534718 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:08.534756 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:08.534942 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:08.535895 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:08.535899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:08.535899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:08.535905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:08.535949 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:08.535966 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:08.536035 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:08.536226 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 05:19:08.536270 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260925 05:19:08.536696 1979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:08.537463 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:08.538480 1979 catalog_manager.cc:1347] Generated new cluster ID: 88f30c6d41fa46839bda3e4aa7ad68e2
I20260925 05:19:08.538539 1979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:08.540546 1968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:08.539237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32963 (local address 127.0.0.1:44957)
0925 05:19:08.539275 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:08.539277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:08.539396 (+ 119us) server_negotiation.cc:407] Connection header received
0925 05:19:08.539499 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:08.539500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:08.539505 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:08.539536 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:08.540283 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:08.540284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:08.540284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:08.540289 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:08.540304 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:08.540309 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:08.540340 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:08.540444 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 05:19:08.540458 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:19:08.561540 1979 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:08.561761 1979 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:08.575876 1979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e59a091b668746a3be09f462cb9f45a8: Generated new TSK 0
I20260925 05:19:08.576138 1979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:08.582556 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 1892
2026-09-25T05:19:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:08 dist-test-slave-9tgz krb5kdc[1992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:08 dist-test-slave-9tgz krb5kdc[1992](info): set up 2 sockets
Sep 25 05:19:08 dist-test-slave-9tgz krb5kdc[1992](info): commencing operation
krb5kdc: starting...
W20260925 05:19:09.611917 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:09 dist-test-slave-9tgz krb5kdc[1992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:09.734030 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:09.819870 2005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:09.820163 2005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:09.820212 2005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:09.821558 2005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:09.821652 2005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:09.821677 2005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:09.821699 2005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:09.821727 2005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:09.823519 2005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:09.823873 2005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:09.824146 2005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:19:09.829890 2005 server_base.cc:1034] running on GCE node
W20260925 05:19:09.829869 2011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:09.829869 2013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:09.829850 2010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:09.830387 2005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:09.830754 2005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:09.831943 2005 hybrid_clock.cc:648] HybridClock initialized: now 1790313549831926 us; error 58 us; skew 500 ppm
Sep 25 05:19:09 dist-test-slave-9tgz krb5kdc[1992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:09.834695 2005 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:09.835332 2005 webserver.cc:460] Webserver started at http://127.0.0.1:35593/ using document root <none> and password file <none>
I20260925 05:19:09.835608 2005 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:09.835685 2005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:09.835841 2005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:09.837080 2005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "6df710d30a224302a1700009956da165"
format_stamp: "Formatted at 2026-09-25 05:19:09 on dist-test-slave-9tgz"
server_key: "e7f5191f467ae8d2768ebb3d090179ad"
server_key_iv: "b8bf4541c80d1792b688bac6cd858788"
server_key_version: "encryptionkey@0"
I20260925 05:19:09.837538 2005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "6df710d30a224302a1700009956da165"
format_stamp: "Formatted at 2026-09-25 05:19:09 on dist-test-slave-9tgz"
server_key: "e7f5191f467ae8d2768ebb3d090179ad"
server_key_iv: "b8bf4541c80d1792b688bac6cd858788"
server_key_version: "encryptionkey@0"
I20260925 05:19:09.839095 2005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:09.839974 2020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:09.840205 2005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:09.840324 2005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "6df710d30a224302a1700009956da165"
format_stamp: "Formatted at 2026-09-25 05:19:09 on dist-test-slave-9tgz"
server_key: "e7f5191f467ae8d2768ebb3d090179ad"
server_key_iv: "b8bf4541c80d1792b688bac6cd858788"
server_key_version: "encryptionkey@0"
I20260925 05:19:09.840411 2005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:09.859277 2005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:09.861893 2005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:09.862097 2005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:09.866489 2005 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33359
I20260925 05:19:09.866529 2073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33359 every 8 connection(s)
I20260925 05:19:09.866947 2005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:09.867920 2074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:09.869911 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2005
I20260925 05:19:09.870054 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:09.870235 13954 external_mini_cluster.cc:1428] Setting key cddf33356c50c2f85ca49117232b5387
I20260925 05:19:09.871009 2074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: Bootstrap starting.
I20260925 05:19:09.871701 2074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:09.872025 2074 log.cc:824] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:09.872874 2074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: No bootstrap required, opened a new log
I20260925 05:19:09.875414 2074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260925 05:19:09.875622 2074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:09.875658 2074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df710d30a224302a1700009956da165, State: Initialized, Role: FOLLOWER
I20260925 05:19:09.875801 2074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260925 05:19:09.875895 2074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:09.875932 2074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:09.875989 2074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:09.876652 2074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260925 05:19:09.876797 2074 leader_election.cc:302] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df710d30a224302a1700009956da165; no voters:
I20260925 05:19:09.877014 2074 leader_election.cc:288] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:09.877084 2079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:09.877265 2079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [term 1 LEADER]: Becoming Leader. State: Replica: 6df710d30a224302a1700009956da165, State: Running, Role: LEADER
I20260925 05:19:09.877352 2074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:09.877455 2079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260925 05:19:09.877880 2080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6df710d30a224302a1700009956da165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } } }
I20260925 05:19:09.877983 2080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:09.877872 2081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df710d30a224302a1700009956da165. Latest consensus state: current_term: 1 leader_uuid: "6df710d30a224302a1700009956da165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df710d30a224302a1700009956da165" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } } }
Sep 25 05:19:09 dist-test-slave-9tgz krb5kdc[1992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 05:19:09.878648 2081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165 [sys.catalog]: This master's current role is: LEADER
W20260925 05:19:09.879539 2095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:09.879627 2095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:09.879740 2086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:09.879984 2086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:09.881852 2086 catalog_manager.cc:1347] Generated new cluster ID: b4c9b268b5a445c48efae082cd2968e8
I20260925 05:19:09.881916 2086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:09.883109 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:09.870845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53221 (local address 127.0.0.1:33359)
0925 05:19:09.871079 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:09.871091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:09.871151 (+ 60us) server_negotiation.cc:407] Connection header received
0925 05:19:09.871648 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:09.871655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:09.871710 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:09.871884 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:09.872687 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:09.874116 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:09.875044 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:09.875735 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:09.875858 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:09.879276 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:09.879282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:09.879283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:09.879299 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:09.881252 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:09.881722 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:09.881727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:09.881727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:09.881853 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:09.882127 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:09.882128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:09.882129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:09.882410 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:09.882599 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:09.882723 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 05:19:09.882884 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":126,"threads_started":1}
I20260925 05:19:09.884097 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:09.893362 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:09.885567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56235 (local address 127.0.0.1:33359)
0925 05:19:09.885610 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:09.885612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:09.885769 (+ 157us) server_negotiation.cc:407] Connection header received
0925 05:19:09.885836 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:09.885837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:09.885846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:09.885892 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:09.886539 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:09.887114 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:09.888167 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:09.888396 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:09.888480 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:09.890038 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:09.890039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:09.890040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:09.890048 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:09.891942 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:09.892451 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:09.892453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:09.892453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:09.892544 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:09.892771 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:09.892772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:09.892773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:09.892927 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:09.893038 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:09.893153 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 05:19:09.893256 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:09.930614 2086 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:09.931275 2086 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:09.935415 2086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6df710d30a224302a1700009956da165: Generated new TSK 0
I20260925 05:19:09.935657 2086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:09.944887 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2005
2026-09-25T05:19:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:10 dist-test-slave-9tgz krb5kdc[2105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:10 dist-test-slave-9tgz krb5kdc[2105](info): set up 2 sockets
Sep 25 05:19:10 dist-test-slave-9tgz krb5kdc[2105](info): commencing operation
krb5kdc: starting...
W20260925 05:19:10.983891 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:11 dist-test-slave-9tgz krb5kdc[2105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:11.105469 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:11.187772 2118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:11.188048 2118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:11.188094 2118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:11.189402 2118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:11.189487 2118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:11.189512 2118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:11.189534 2118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:11.189560 2118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:11.191205 2118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:11.191506 2118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:11.191758 2118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:11.196931 2123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:11.196933 2126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:11.196971 2124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:11.197218 2118 server_base.cc:1034] running on GCE node
I20260925 05:19:11.197511 2118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:11.197883 2118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:11.199034 2118 hybrid_clock.cc:648] HybridClock initialized: now 1790313551199008 us; error 39 us; skew 500 ppm
Sep 25 05:19:11 dist-test-slave-9tgz krb5kdc[2105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313551, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:11.201845 2118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:11.202303 2118 webserver.cc:460] Webserver started at http://127.0.0.1:44647/ using document root <none> and password file <none>
I20260925 05:19:11.202524 2118 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:11.202569 2118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:11.202674 2118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:11.204192 2118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "a95694ee8ab344b5bf681806d9da140f"
format_stamp: "Formatted at 2026-09-25 05:19:11 on dist-test-slave-9tgz"
server_key: "20f414fe1bf52cc1311abf966d06df7c"
server_key_iv: "a8eccb75af6ba5b6df58386f8d5a777a"
server_key_version: "encryptionkey@0"
I20260925 05:19:11.204543 2118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "a95694ee8ab344b5bf681806d9da140f"
format_stamp: "Formatted at 2026-09-25 05:19:11 on dist-test-slave-9tgz"
server_key: "20f414fe1bf52cc1311abf966d06df7c"
server_key_iv: "a8eccb75af6ba5b6df58386f8d5a777a"
server_key_version: "encryptionkey@0"
I20260925 05:19:11.206001 2118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:11.206835 2133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:11.207062 2118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:11.207190 2118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "a95694ee8ab344b5bf681806d9da140f"
format_stamp: "Formatted at 2026-09-25 05:19:11 on dist-test-slave-9tgz"
server_key: "20f414fe1bf52cc1311abf966d06df7c"
server_key_iv: "a8eccb75af6ba5b6df58386f8d5a777a"
server_key_version: "encryptionkey@0"
I20260925 05:19:11.207280 2118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:11.218809 2118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:11.221511 2118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:11.221716 2118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:11.226032 2118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39897
I20260925 05:19:11.226086 2185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39897 every 8 connection(s)
I20260925 05:19:11.226509 2118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:11.227490 2186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:11.229905 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: Bootstrap starting.
I20260925 05:19:11.230546 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2118
I20260925 05:19:11.230574 2186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:11.230690 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:11.230865 13954 external_mini_cluster.cc:1428] Setting key 0ade3ed431df06eb1b3095bc472cf556
I20260925 05:19:11.230952 2186 log.cc:824] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:11.231875 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: No bootstrap required, opened a new log
I20260925 05:19:11.233593 2186 raft_consensus.cc:348] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 05:19:11.233767 2186 raft_consensus.cc:374] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:11.233793 2186 raft_consensus.cc:729] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95694ee8ab344b5bf681806d9da140f, State: Initialized, Role: FOLLOWER
I20260925 05:19:11.233951 2186 consensus_queue.cc:260] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 05:19:11.234069 2186 raft_consensus.cc:388] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:11.234115 2186 raft_consensus.cc:482] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:11.234171 2186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:11.235111 2186 raft_consensus.cc:504] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 05:19:11.235226 2186 leader_election.cc:302] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a95694ee8ab344b5bf681806d9da140f; no voters:
I20260925 05:19:11.235414 2186 leader_election.cc:288] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:11.235504 2191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:11.235690 2191 raft_consensus.cc:686] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [term 1 LEADER]: Becoming Leader. State: Replica: a95694ee8ab344b5bf681806d9da140f, State: Running, Role: LEADER
I20260925 05:19:11.235702 2186 sys_catalog.cc:564] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:11.235857 2191 consensus_queue.cc:237] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260925 05:19:11.237879 2192 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a95694ee8ab344b5bf681806d9da140f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260925 05:19:11.238468 2192 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:11.238036 2193 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a95694ee8ab344b5bf681806d9da140f. Latest consensus state: current_term: 1 leader_uuid: "a95694ee8ab344b5bf681806d9da140f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95694ee8ab344b5bf681806d9da140f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260925 05:19:11.238818 2193 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f [sys.catalog]: This master's current role is: LEADER
W20260925 05:19:11.238847 2206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:11.238893 2206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:11.239732 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:11.231447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42847 (local address 127.0.0.1:39897)
0925 05:19:11.231676 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:11.231687 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:11.234027 (+ 2340us) server_negotiation.cc:407] Connection header received
0925 05:19:11.234279 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:11.234287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:11.234352 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:11.234527 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:11.234988 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:11.236353 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:11.237441 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:11.238072 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:11.238216 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:11.239053 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:11.239058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:11.239059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:11.239247 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:11.239280 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:11.239292 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:11.239376 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:11.239459 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 05:19:11.239529 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
I20260925 05:19:11.240820 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:11.242975 2209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:11.243280 2209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:11.245230 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:11.242202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37789 (local address 127.0.0.1:39897)
0925 05:19:11.242252 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:11.242254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:11.242263 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:11.242317 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:11.242318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:11.242327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:11.242378 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:11.242522 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:11.243097 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:11.243918 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:11.244136 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:11.244213 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:11.244966 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:11.244967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:11.244968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:11.244973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:11.244988 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:11.244993 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:11.245049 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:11.245117 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 05:19:11.245145 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 05:19:11.245467 2209 catalog_manager.cc:1347] Generated new cluster ID: 64fb839197ec43a0b3d08151747e0ae1
I20260925 05:19:11.245518 2209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:11.255915 2209 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:11.256695 2209 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:11.265657 2209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a95694ee8ab344b5bf681806d9da140f: Generated new TSK 0
I20260925 05:19:11.265923 2209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:11.274101 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2118
2026-09-25T05:19:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:11 dist-test-slave-9tgz krb5kdc[2218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:11 dist-test-slave-9tgz krb5kdc[2218](info): set up 2 sockets
Sep 25 05:19:11 dist-test-slave-9tgz krb5kdc[2218](info): commencing operation
krb5kdc: starting...
W20260925 05:19:12.320106 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:12 dist-test-slave-9tgz krb5kdc[2218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:12.445680 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:12.529964 2231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:12.530215 2231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:12.530241 2231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:12.531471 2231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:12.531518 2231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:12.531533 2231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:12.531545 2231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:12.531558 2231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:12.533063 2231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:12.533288 2231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:12.533483 2231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:12.538718 2237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:12.538765 2231 server_base.cc:1034] running on GCE node
W20260925 05:19:12.538709 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:12.538772 2236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:12.539243 2231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:12.539553 2231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:12.540733 2231 hybrid_clock.cc:648] HybridClock initialized: now 1790313552540709 us; error 42 us; skew 500 ppm
Sep 25 05:19:12 dist-test-slave-9tgz krb5kdc[2218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:12.543682 2231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:12.544167 2231 webserver.cc:460] Webserver started at http://127.0.0.1:39841/ using document root <none> and password file <none>
I20260925 05:19:12.544402 2231 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:12.544456 2231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:12.544574 2231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:12.545758 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "11397735bd9c44be8a09f23105ff951d"
format_stamp: "Formatted at 2026-09-25 05:19:12 on dist-test-slave-9tgz"
server_key: "0d1ca9a9e9c5a62c0c3738b75d99487c"
server_key_iv: "e86debb0da670097ae80b1c7bf9d4e72"
server_key_version: "encryptionkey@0"
I20260925 05:19:12.546111 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "11397735bd9c44be8a09f23105ff951d"
format_stamp: "Formatted at 2026-09-25 05:19:12 on dist-test-slave-9tgz"
server_key: "0d1ca9a9e9c5a62c0c3738b75d99487c"
server_key_iv: "e86debb0da670097ae80b1c7bf9d4e72"
server_key_version: "encryptionkey@0"
I20260925 05:19:12.547684 2231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:12.548532 2246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:12.548754 2231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:12.548882 2231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "11397735bd9c44be8a09f23105ff951d"
format_stamp: "Formatted at 2026-09-25 05:19:12 on dist-test-slave-9tgz"
server_key: "0d1ca9a9e9c5a62c0c3738b75d99487c"
server_key_iv: "e86debb0da670097ae80b1c7bf9d4e72"
server_key_version: "encryptionkey@0"
I20260925 05:19:12.548975 2231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:12.552768 2231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:12.555480 2231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:12.555680 2231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:12.560007 2231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36875
I20260925 05:19:12.560056 2298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36875 every 8 connection(s)
I20260925 05:19:12.560447 2231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:12.560804 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2231
I20260925 05:19:12.560949 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:12.561169 13954 external_mini_cluster.cc:1428] Setting key 27368383c3ef8c06261d129d77b36256
I20260925 05:19:12.561661 2299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:12.563953 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:12.561742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56863 (local address 127.0.0.1:36875)
0925 05:19:12.561962 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:12.561970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:12.562003 (+ 33us) server_negotiation.cc:407] Connection header received
0925 05:19:12.562464 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:12.562472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:12.562507 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:12.562641 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:12.563433 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:12.563437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:12.563438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:12.563442 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:12.563487 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:12.563506 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:12.563569 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:12.563660 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 05:19:12.563722 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":123,"threads_started":1}
I20260925 05:19:12.565109 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:12.565476 2299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d: Bootstrap starting.
I20260925 05:19:12.566319 2299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:12.566707 2299 log.cc:824] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:12.567673 2299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d: No bootstrap required, opened a new log
I20260925 05:19:12.568374 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:12.566923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34253 (local address 127.0.0.1:36875)
0925 05:19:12.566972 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:12.566975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:12.567215 (+ 240us) server_negotiation.cc:407] Connection header received
0925 05:19:12.567326 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:12.567327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:12.567332 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:12.567386 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:12.568087 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:12.568088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:12.568088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:12.568092 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:12.568106 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:12.568111 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:12.568174 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:12.568261 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 05:19:12.568273 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 05:19:12.569973 2299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 05:19:12.570163 2299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:12.570218 2299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11397735bd9c44be8a09f23105ff951d, State: Initialized, Role: FOLLOWER
I20260925 05:19:12.570353 2299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 05:19:12.570422 2299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:12.570458 2299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:12.570489 2299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:12.571235 2299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 05:19:12.571373 2299 leader_election.cc:302] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11397735bd9c44be8a09f23105ff951d; no voters:
I20260925 05:19:12.571594 2299 leader_election.cc:288] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:12.571677 2309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:12.571848 2309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [term 1 LEADER]: Becoming Leader. State: Replica: 11397735bd9c44be8a09f23105ff951d, State: Running, Role: LEADER
I20260925 05:19:12.571908 2299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:12.571992 2309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } }
I20260925 05:19:12.572434 2310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11397735bd9c44be8a09f23105ff951d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } } }
I20260925 05:19:12.572544 2310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:12.572747 2311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11397735bd9c44be8a09f23105ff951d. Latest consensus state: current_term: 1 leader_uuid: "11397735bd9c44be8a09f23105ff951d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11397735bd9c44be8a09f23105ff951d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36875 } } }
I20260925 05:19:12.572824 2311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:12.573180 2318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:12.573725 2318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:12.575446 2318 catalog_manager.cc:1347] Generated new cluster ID: 695b3e4240704fea83a9afe9ee63715d
I20260925 05:19:12.575507 2318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:12.594391 2318 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:12.594645 2318 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:12.598501 2318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11397735bd9c44be8a09f23105ff951d: Generated new TSK 0
I20260925 05:19:12.598740 2318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:12.601833 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2231
2026-09-25T05:19:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:12 dist-test-slave-9tgz krb5kdc[2330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:12 dist-test-slave-9tgz krb5kdc[2330](info): set up 2 sockets
Sep 25 05:19:12 dist-test-slave-9tgz krb5kdc[2330](info): commencing operation
krb5kdc: starting...
W20260925 05:19:13.655982 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:13 dist-test-slave-9tgz krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:13.778134 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:13.862722 2343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:13.863063 2343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:13.863103 2343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:13.864439 2343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:13.864497 2343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:13.864512 2343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:13.864526 2343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:13.864586 2343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:13.866341 2343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:13.866652 2343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:13.866931 2343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:13.871970 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:13.871970 2351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:13.871978 2349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:13.872318 2343 server_base.cc:1034] running on GCE node
I20260925 05:19:13.872613 2343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:13.872973 2343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:13.874156 2343 hybrid_clock.cc:648] HybridClock initialized: now 1790313553874131 us; error 56 us; skew 500 ppm
Sep 25 05:19:13 dist-test-slave-9tgz krb5kdc[2330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313553, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:13.882834 2343 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:13.883446 2343 webserver.cc:460] Webserver started at http://127.0.0.1:38301/ using document root <none> and password file <none>
I20260925 05:19:13.883733 2343 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:13.883808 2343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:13.883965 2343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:13.885113 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "706be08dd9c84d4b9ba2fa9fb56160e7"
format_stamp: "Formatted at 2026-09-25 05:19:13 on dist-test-slave-9tgz"
server_key: "674cda8f551e3839f319e154569508a1"
server_key_iv: "70dabe95783bb8690d2b0b29279a810d"
server_key_version: "encryptionkey@0"
I20260925 05:19:13.885507 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "706be08dd9c84d4b9ba2fa9fb56160e7"
format_stamp: "Formatted at 2026-09-25 05:19:13 on dist-test-slave-9tgz"
server_key: "674cda8f551e3839f319e154569508a1"
server_key_iv: "70dabe95783bb8690d2b0b29279a810d"
server_key_version: "encryptionkey@0"
I20260925 05:19:13.887012 2343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 05:19:13.887845 2358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:13.888019 2343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:13.888126 2343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "706be08dd9c84d4b9ba2fa9fb56160e7"
format_stamp: "Formatted at 2026-09-25 05:19:13 on dist-test-slave-9tgz"
server_key: "674cda8f551e3839f319e154569508a1"
server_key_iv: "70dabe95783bb8690d2b0b29279a810d"
server_key_version: "encryptionkey@0"
I20260925 05:19:13.888213 2343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:13.905503 2343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:13.907697 2343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:13.907891 2343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:13.912247 2343 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41709
I20260925 05:19:13.912284 2411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41709 every 8 connection(s)
I20260925 05:19:13.912685 2343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:13.913224 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2343
I20260925 05:19:13.913345 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:13.913550 13954 external_mini_cluster.cc:1428] Setting key 4d66f0a57f341213d933cb7e7cbf228b
I20260925 05:19:13.913757 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:13.917356 2412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: Bootstrap starting.
I20260925 05:19:13.918118 2412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:13.918439 2412 log.cc:824] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:13.919237 2412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: No bootstrap required, opened a new log
Sep 25 05:19:13 dist-test-slave-9tgz krb5kdc[2330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 05:19:13.921365 2412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260925 05:19:13.921573 2412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:13.921655 2412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706be08dd9c84d4b9ba2fa9fb56160e7, State: Initialized, Role: FOLLOWER
I20260925 05:19:13.921844 2412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260925 05:19:13.921952 2412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:13.921984 2412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:13.922039 2412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:13.922922 2412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260925 05:19:13.923043 2412 leader_election.cc:302] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 706be08dd9c84d4b9ba2fa9fb56160e7; no voters:
I20260925 05:19:13.923242 2412 leader_election.cc:288] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:13.923321 2417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:13.923499 2417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [term 1 LEADER]: Becoming Leader. State: Replica: 706be08dd9c84d4b9ba2fa9fb56160e7, State: Running, Role: LEADER
I20260925 05:19:13.923527 2412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:13.923595 2417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260925 05:19:13.924018 2418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } } }
I20260925 05:19:13.924008 2417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706be08dd9c84d4b9ba2fa9fb56160e7. Latest consensus state: current_term: 1 leader_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706be08dd9c84d4b9ba2fa9fb56160e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } } }
I20260925 05:19:13.924163 2418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:13.924171 2417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:13.925402 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:13.914192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:50055 (local address 127.0.0.1:41709)
0925 05:19:13.914379 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:13.914389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:13.914426 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:19:13.915112 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:13.915121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:13.915182 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:13.915316 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:13.916174 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:13.917093 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:13.918197 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:13.918674 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:13.918765 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:13.921182 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:13.921187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:13.921187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:13.921200 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:13.923280 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:13.923811 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:13.923814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:13.923814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:13.923919 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:13.924229 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:13.924230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:13.924230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:13.924584 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:13.924770 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:13.924918 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 05:19:13.925165 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260925 05:19:13.925531 2432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:13.925594 2432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:13.926035 2426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:13.926265 2426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:13.926662 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:13.927978 2426 catalog_manager.cc:1347] Generated new cluster ID: fc44431cf03d43b7ba21ad2e7e7e4cb5
I20260925 05:19:13.928059 2426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:13.936264 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:13.928312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:54447 (local address 127.0.0.1:41709)
0925 05:19:13.928392 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:13.928395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:13.928518 (+ 123us) server_negotiation.cc:407] Connection header received
0925 05:19:13.928566 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:13.928567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:13.928579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:13.928642 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:13.929305 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:13.930069 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:13.931155 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:13.931513 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:13.931594 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:13.933120 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:13.933122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:13.933122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:13.933131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:13.934919 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:13.935363 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:13.935365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:13.935365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:13.935436 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:13.935680 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:13.935681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:13.935681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:13.935838 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:13.935952 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:13.936072 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 05:19:13.936158 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 05:19:13.940460 2426 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:13.941009 2426 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:13.943398 2426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 706be08dd9c84d4b9ba2fa9fb56160e7: Generated new TSK 0
I20260925 05:19:13.943591 2426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:13.947929 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2343
W20260925 05:19:13.952468 2328 connection.cc:535] client connection to 127.0.0.1:41709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41709): Connection reset by peer (error 104)
2026-09-25T05:19:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:14 dist-test-slave-9tgz krb5kdc[2443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:14 dist-test-slave-9tgz krb5kdc[2443](info): set up 2 sockets
Sep 25 05:19:14 dist-test-slave-9tgz krb5kdc[2443](info): commencing operation
krb5kdc: starting...
W20260925 05:19:14.979756 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:15 dist-test-slave-9tgz krb5kdc[2443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:15.097345 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:15.178915 2457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:15.179121 2457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:15.179144 2457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:15.180336 2457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:15.180379 2457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:15.180393 2457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:15.180405 2457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:15.180418 2457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:15.181921 2457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:15.182193 2457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:15.182472 2457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:15.187247 2465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:15.187212 2462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:15.187400 2457 server_base.cc:1034] running on GCE node
W20260925 05:19:15.187209 2463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:15.187664 2457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:15.187912 2457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:15.189050 2457 hybrid_clock.cc:648] HybridClock initialized: now 1790313555189042 us; error 27 us; skew 500 ppm
Sep 25 05:19:15 dist-test-slave-9tgz krb5kdc[2443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:15.191504 2457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:15.192014 2457 webserver.cc:460] Webserver started at http://127.0.0.1:41261/ using document root <none> and password file <none>
I20260925 05:19:15.192265 2457 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:15.192335 2457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:15.192483 2457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:15.193533 2457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "009f834ef539492aab55cbe1d85cc542"
format_stamp: "Formatted at 2026-09-25 05:19:15 on dist-test-slave-9tgz"
server_key: "5615ad588c8aa1b604103581bd551775"
server_key_iv: "0ee1553b82965468d9b35919b78c2eb6"
server_key_version: "encryptionkey@0"
I20260925 05:19:15.193890 2457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "009f834ef539492aab55cbe1d85cc542"
format_stamp: "Formatted at 2026-09-25 05:19:15 on dist-test-slave-9tgz"
server_key: "5615ad588c8aa1b604103581bd551775"
server_key_iv: "0ee1553b82965468d9b35919b78c2eb6"
server_key_version: "encryptionkey@0"
I20260925 05:19:15.195228 2457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:19:15.195931 2472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:15.196100 2457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:19:15.196215 2457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "009f834ef539492aab55cbe1d85cc542"
format_stamp: "Formatted at 2026-09-25 05:19:15 on dist-test-slave-9tgz"
server_key: "5615ad588c8aa1b604103581bd551775"
server_key_iv: "0ee1553b82965468d9b35919b78c2eb6"
server_key_version: "encryptionkey@0"
I20260925 05:19:15.196292 2457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:15.214830 2457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:15.216872 2457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:15.217054 2457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:15.221242 2457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32821
I20260925 05:19:15.221289 2524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32821 every 8 connection(s)
I20260925 05:19:15.221673 2457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:15.222301 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2457
I20260925 05:19:15.222424 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:15.222622 13954 external_mini_cluster.cc:1428] Setting key 7c3f8772a6a08b9c2e3a1fab977f3d5f
I20260925 05:19:15.222760 2525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:15.226065 2525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542: Bootstrap starting.
I20260925 05:19:15.226711 2525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:15.227080 2525 log.cc:824] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:15.227887 2525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542: No bootstrap required, opened a new log
I20260925 05:19:15.228395 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:15.223210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:37945 (local address 127.0.0.1:32821)
0925 05:19:15.223404 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:15.223413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:15.223453 (+ 40us) server_negotiation.cc:407] Connection header received
0925 05:19:15.223959 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:15.223966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:15.224015 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:15.224126 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:15.224353 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:15.225227 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:15.226330 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:15.226724 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:15.226867 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:15.227694 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:15.227698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:15.227698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:15.227856 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:15.227885 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:15.227896 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:15.228000 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:15.228128 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 05:19:15.228173 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260925 05:19:15.229436 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:15.230360 2525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260925 05:19:15.230545 2525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:15.230616 2525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009f834ef539492aab55cbe1d85cc542, State: Initialized, Role: FOLLOWER
I20260925 05:19:15.230763 2525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260925 05:19:15.230855 2525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:15.230909 2525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:15.230962 2525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:15.231735 2525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260925 05:19:15.231863 2525 leader_election.cc:302] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 009f834ef539492aab55cbe1d85cc542; no voters:
I20260925 05:19:15.232012 2525 leader_election.cc:288] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:15.232067 2535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:15.232193 2535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [term 1 LEADER]: Becoming Leader. State: Replica: 009f834ef539492aab55cbe1d85cc542, State: Running, Role: LEADER
I20260925 05:19:15.232295 2525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:15.232326 2535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260925 05:19:15.232652 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "009f834ef539492aab55cbe1d85cc542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260925 05:19:15.232736 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:15.232640 2535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 009f834ef539492aab55cbe1d85cc542. Latest consensus state: current_term: 1 leader_uuid: "009f834ef539492aab55cbe1d85cc542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009f834ef539492aab55cbe1d85cc542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260925 05:19:15.233251 2535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:15.233443 2541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:15.233932 2541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:15.234597 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:15.231027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:51623 (local address 127.0.0.1:32821)
0925 05:19:15.231063 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:15.231066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:15.231242 (+ 176us) server_negotiation.cc:407] Connection header received
0925 05:19:15.231298 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:15.231299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:15.231307 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:15.231350 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:15.231477 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:15.231973 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:15.232989 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:15.233356 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:15.233402 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:15.234309 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:15.234311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:15.234311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:15.234316 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:15.234330 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:15.234334 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:15.234365 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:15.234497 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 05:19:15.234506 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 05:19:15.235435 2541 catalog_manager.cc:1347] Generated new cluster ID: 4e853e318d9943498cfd9726684c862a
I20260925 05:19:15.235495 2541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:15.242862 2541 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:15.243433 2541 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:15.252573 2541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 009f834ef539492aab55cbe1d85cc542: Generated new TSK 0
I20260925 05:19:15.252806 2541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:15.261929 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2457
W20260925 05:19:15.266017 2441 connection.cc:535] client connection to 127.0.0.1:32821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32821): Connection reset by peer (error 104)
2026-09-25T05:19:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:15 dist-test-slave-9tgz krb5kdc[2556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:15 dist-test-slave-9tgz krb5kdc[2556](info): set up 2 sockets
Sep 25 05:19:15 dist-test-slave-9tgz krb5kdc[2556](info): commencing operation
krb5kdc: starting...
W20260925 05:19:16.299916 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:16 dist-test-slave-9tgz krb5kdc[2556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:16.425869 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:16.510835 2569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:16.511153 2569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:16.511199 2569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:16.512498 2569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:16.512579 2569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:16.512602 2569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:16.512641 2569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:16.512673 2569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:16.514249 2569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:16.514564 2569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:16.514828 2569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:16.519897 2574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:16.519901 2575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:16.519897 2577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:16.520226 2569 server_base.cc:1034] running on GCE node
I20260925 05:19:16.520471 2569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:16.520838 2569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:16.522014 2569 hybrid_clock.cc:648] HybridClock initialized: now 1790313556522008 us; error 38 us; skew 500 ppm
Sep 25 05:19:16 dist-test-slave-9tgz krb5kdc[2556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313556, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:16.524677 2569 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:16.525177 2569 webserver.cc:460] Webserver started at http://127.0.0.1:41159/ using document root <none> and password file <none>
I20260925 05:19:16.525417 2569 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:16.525490 2569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:16.525656 2569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:16.526849 2569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9"
format_stamp: "Formatted at 2026-09-25 05:19:16 on dist-test-slave-9tgz"
server_key: "06d69513738abafc51ee28b688e5cf20"
server_key_iv: "1127e84ee3505b732cd139833b5e9e63"
server_key_version: "encryptionkey@0"
I20260925 05:19:16.527308 2569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9"
format_stamp: "Formatted at 2026-09-25 05:19:16 on dist-test-slave-9tgz"
server_key: "06d69513738abafc51ee28b688e5cf20"
server_key_iv: "1127e84ee3505b732cd139833b5e9e63"
server_key_version: "encryptionkey@0"
I20260925 05:19:16.528761 2569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:16.529627 2584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:16.529805 2569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:16.529917 2569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9"
format_stamp: "Formatted at 2026-09-25 05:19:16 on dist-test-slave-9tgz"
server_key: "06d69513738abafc51ee28b688e5cf20"
server_key_iv: "1127e84ee3505b732cd139833b5e9e63"
server_key_version: "encryptionkey@0"
I20260925 05:19:16.530001 2569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:16.533573 2569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:16.535722 2569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:16.535894 2569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:16.539921 2569 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46233
I20260925 05:19:16.539974 2637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46233 every 8 connection(s)
I20260925 05:19:16.540369 2569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:16.540956 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2569
I20260925 05:19:16.541085 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:16.541316 13954 external_mini_cluster.cc:1428] Setting key 2cfcbf3959a090d67bc4029ca2cfe50a
I20260925 05:19:16.541554 2638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:16.544111 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:16.542008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:55973 (local address 127.0.0.1:46233)
0925 05:19:16.542210 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:16.542218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:16.542243 (+ 25us) server_negotiation.cc:407] Connection header received
0925 05:19:16.542691 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:16.542698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:16.542726 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:16.542852 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:16.543618 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:16.543622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:16.543622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:16.543625 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:16.543669 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:16.543680 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:16.543738 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:16.543834 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 05:19:16.543896 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260925 05:19:16.545195 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9: Bootstrap starting.
I20260925 05:19:16.545256 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:16.546025 2638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:16.546417 2638 log.cc:824] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:16.547470 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9: No bootstrap required, opened a new log
I20260925 05:19:16.548188 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:16.547037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:34841 (local address 127.0.0.1:46233)
0925 05:19:16.547096 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:16.547098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:16.547250 (+ 152us) server_negotiation.cc:407] Connection header received
0925 05:19:16.547285 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:16.547286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:16.547292 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:16.547367 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:16.547966 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:16.547967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:16.547968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:16.547972 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:16.547989 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:16.547993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:16.548062 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:16.548106 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 05:19:16.548120 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 05:19:16.549863 2638 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } }
I20260925 05:19:16.550130 2638 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:16.550192 2638 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ea7c3dd6f94fef9e76f87709a6e5c9, State: Initialized, Role: FOLLOWER
I20260925 05:19:16.550333 2638 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } }
I20260925 05:19:16.550415 2638 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:16.550477 2638 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:16.550529 2638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:16.551230 2638 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } }
I20260925 05:19:16.551353 2638 leader_election.cc:302] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2ea7c3dd6f94fef9e76f87709a6e5c9; no voters:
I20260925 05:19:16.551563 2638 leader_election.cc:288] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:16.551661 2648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:16.551839 2648 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [term 1 LEADER]: Becoming Leader. State: Replica: f2ea7c3dd6f94fef9e76f87709a6e5c9, State: Running, Role: LEADER
I20260925 05:19:16.551994 2648 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } }
I20260925 05:19:16.551834 2638 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:16.552378 2649 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } } }
I20260925 05:19:16.552465 2649 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:16.552556 2650 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2ea7c3dd6f94fef9e76f87709a6e5c9. Latest consensus state: current_term: 1 leader_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2ea7c3dd6f94fef9e76f87709a6e5c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46233 } } }
I20260925 05:19:16.552829 2650 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:16.552841 2652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:16.553541 2652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:16.555375 2652 catalog_manager.cc:1347] Generated new cluster ID: ce45fa0fa7d544d1859369cf5bee562c
I20260925 05:19:16.555457 2652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:16.580824 2652 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:16.581063 2652 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:16.590862 2652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2ea7c3dd6f94fef9e76f87709a6e5c9: Generated new TSK 0
I20260925 05:19:16.591159 2652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:16.600111 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2569
2026-09-25T05:19:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:16 dist-test-slave-9tgz krb5kdc[2669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:16 dist-test-slave-9tgz krb5kdc[2669](info): set up 2 sockets
Sep 25 05:19:16 dist-test-slave-9tgz krb5kdc[2669](info): commencing operation
krb5kdc: starting...
W20260925 05:19:17.635989 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:17.757483 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:17.839531 2682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:17.839751 2682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:17.839774 2682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:17.840963 2682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:17.841013 2682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:17.841028 2682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:17.841040 2682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:17.841099 2682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:17.842612 2682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:17.842880 2682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:17.843150 2682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:17.848053 2687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:17.848133 2682 server_base.cc:1034] running on GCE node
W20260925 05:19:17.848038 2688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:17.848038 2690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:17.848536 2682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:17.848886 2682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:17.850036 2682 hybrid_clock.cc:648] HybridClock initialized: now 1790313557850017 us; error 46 us; skew 500 ppm
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:17.852550 2682 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:17.853024 2682 webserver.cc:460] Webserver started at http://127.0.0.1:43015/ using document root <none> and password file <none>
I20260925 05:19:17.853274 2682 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:17.853343 2682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:17.853488 2682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:17.854566 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "7b952d41e46142ebaf0f91bac251ac8b"
format_stamp: "Formatted at 2026-09-25 05:19:17 on dist-test-slave-9tgz"
server_key: "7fdbb39156d351cbddac6da8890b48b3"
server_key_iv: "2c0677db88dcc024a6493e426ae389f8"
server_key_version: "encryptionkey@0"
I20260925 05:19:17.854992 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "7b952d41e46142ebaf0f91bac251ac8b"
format_stamp: "Formatted at 2026-09-25 05:19:17 on dist-test-slave-9tgz"
server_key: "7fdbb39156d351cbddac6da8890b48b3"
server_key_iv: "2c0677db88dcc024a6493e426ae389f8"
server_key_version: "encryptionkey@0"
I20260925 05:19:17.856303 2682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:19:17.856998 2697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:17.857178 2682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:19:17.857267 2682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "7b952d41e46142ebaf0f91bac251ac8b"
format_stamp: "Formatted at 2026-09-25 05:19:17 on dist-test-slave-9tgz"
server_key: "7fdbb39156d351cbddac6da8890b48b3"
server_key_iv: "2c0677db88dcc024a6493e426ae389f8"
server_key_version: "encryptionkey@0"
I20260925 05:19:17.857350 2682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:17.871009 2682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:17.872925 2682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:17.873103 2682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:17.877327 2682 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35591
I20260925 05:19:17.877365 2749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35591 every 8 connection(s)
I20260925 05:19:17.877759 2682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:17.878669 2750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:17.881057 2750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b: Bootstrap starting.
I20260925 05:19:17.881609 2750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:17.881841 2750 log.cc:824] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:17.882478 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2682
I20260925 05:19:17.882588 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:17.882674 2750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b: No bootstrap required, opened a new log
I20260925 05:19:17.882786 13954 external_mini_cluster.cc:1428] Setting key 55f199bb7cf97be1f7864782a3216299
I20260925 05:19:17.885026 2750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } }
I20260925 05:19:17.885210 2750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:17.885244 2750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b952d41e46142ebaf0f91bac251ac8b, State: Initialized, Role: FOLLOWER
I20260925 05:19:17.885387 2750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } }
I20260925 05:19:17.885473 2750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:17.885514 2750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:17.885561 2750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:17.886282 2750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } }
I20260925 05:19:17.886412 2750 leader_election.cc:302] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b952d41e46142ebaf0f91bac251ac8b; no voters:
I20260925 05:19:17.886572 2750 leader_election.cc:288] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:17.886627 2755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:17.886812 2755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [term 1 LEADER]: Becoming Leader. State: Replica: 7b952d41e46142ebaf0f91bac251ac8b, State: Running, Role: LEADER
I20260925 05:19:17.886945 2755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } }
I20260925 05:19:17.886806 2750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:17.887374 2756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b952d41e46142ebaf0f91bac251ac8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } } }
I20260925 05:19:17.887390 2757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b952d41e46142ebaf0f91bac251ac8b. Latest consensus state: current_term: 1 leader_uuid: "7b952d41e46142ebaf0f91bac251ac8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b952d41e46142ebaf0f91bac251ac8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35591 } } }
I20260925 05:19:17.887632 2756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:17.887645 2757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:17.887892 2761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:17.888505 2761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:17.890195 2761 catalog_manager.cc:1347] Generated new cluster ID: 5cd3816fa2344747b199211de72ef5f5
I20260925 05:19:17.890249 2761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 05:19:17.894757 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:17.883331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:52417 (local address 127.0.0.1:35591)
0925 05:19:17.883554 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:17.883564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:17.883598 (+ 34us) server_negotiation.cc:407] Connection header received
0925 05:19:17.884222 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:17.884232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:17.884299 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:17.884455 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:17.885035 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:17.886759 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:17.887825 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:17.888481 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:17.888985 (+ 504us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:17.891337 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:17.891341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:17.891341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:17.891354 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:17.893102 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:17.893503 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:17.893507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:17.893507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:17.893597 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:17.893854 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:17.893855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:17.893855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:17.894096 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:17.894284 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:17.894369 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 05:19:17.894473 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260925 05:19:17.895711 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:17.903414 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:17.897078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:51475 (local address 127.0.0.1:35591)
0925 05:19:17.897115 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:17.897117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:17.897285 (+ 168us) server_negotiation.cc:407] Connection header received
0925 05:19:17.897330 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:17.897331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:17.897342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:17.897389 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:17.897916 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:17.898426 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:17.899264 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:17.899498 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:17.899582 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:17.900776 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:17.900778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:17.900778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:17.900785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:17.902209 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:17.902571 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:17.902572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:17.902572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:17.902626 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:17.902855 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:17.902856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:17.902857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:17.903006 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:17.903119 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:17.903250 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 05:19:17.903312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 05:19:17.922411 2761 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:17.923046 2761 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:17.935868 2761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b952d41e46142ebaf0f91bac251ac8b: Generated new TSK 0
I20260925 05:19:17.936100 2761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:17.940914 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2682
W20260925 05:19:17.945149 2667 connection.cc:535] client connection to 127.0.0.1:35591 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35591): Connection reset by peer (error 104)
2026-09-25T05:19:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2781](info): set up 2 sockets
Sep 25 05:19:17 dist-test-slave-9tgz krb5kdc[2781](info): commencing operation
krb5kdc: starting...
W20260925 05:19:18.967962 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:19 dist-test-slave-9tgz krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:19.086570 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:19.170568 2795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:19.170822 2795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:19.170862 2795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:19.172053 2795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:19.172128 2795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:19.172156 2795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:19.172189 2795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:19.172214 2795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:19.173720 2795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:19.174021 2795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:19.174257 2795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:19.179133 2800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:19.179306 2795 server_base.cc:1034] running on GCE node
W20260925 05:19:19.179133 2801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:19.179155 2803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:19.179733 2795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:19.180038 2795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:19.181205 2795 hybrid_clock.cc:648] HybridClock initialized: now 1790313559181201 us; error 36 us; skew 500 ppm
Sep 25 05:19:19 dist-test-slave-9tgz krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313559, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:19.189217 2795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:19.189702 2795 webserver.cc:460] Webserver started at http://127.0.0.1:33333/ using document root <none> and password file <none>
I20260925 05:19:19.189953 2795 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:19.190025 2795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:19.190178 2795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:19.191428 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "877bec05db364b64b693ab12e35a5bfd"
format_stamp: "Formatted at 2026-09-25 05:19:19 on dist-test-slave-9tgz"
server_key: "0659d37fa705124198261579e615f7f3"
server_key_iv: "92cfa13f4384a0ce0a24c0e5568911f9"
server_key_version: "encryptionkey@0"
I20260925 05:19:19.191790 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "877bec05db364b64b693ab12e35a5bfd"
format_stamp: "Formatted at 2026-09-25 05:19:19 on dist-test-slave-9tgz"
server_key: "0659d37fa705124198261579e615f7f3"
server_key_iv: "92cfa13f4384a0ce0a24c0e5568911f9"
server_key_version: "encryptionkey@0"
I20260925 05:19:19.193109 2795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:19.193825 2810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:19.193995 2795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:19.194103 2795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "877bec05db364b64b693ab12e35a5bfd"
format_stamp: "Formatted at 2026-09-25 05:19:19 on dist-test-slave-9tgz"
server_key: "0659d37fa705124198261579e615f7f3"
server_key_iv: "92cfa13f4384a0ce0a24c0e5568911f9"
server_key_version: "encryptionkey@0"
I20260925 05:19:19.194197 2795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:19.206524 2795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:19.208484 2795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:19.208658 2795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:19.212939 2795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35705
I20260925 05:19:19.212970 2862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35705 every 8 connection(s)
I20260925 05:19:19.213333 2795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:19.214186 2863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:19.216568 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd: Bootstrap starting.
I20260925 05:19:19.217127 2863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:19.217363 2863 log.cc:824] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:19.218020 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd: No bootstrap required, opened a new log
I20260925 05:19:19.219718 2863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260925 05:19:19.219862 2863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:19.219885 2863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877bec05db364b64b693ab12e35a5bfd, State: Initialized, Role: FOLLOWER
I20260925 05:19:19.220026 2863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260925 05:19:19.220109 2863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:19.220134 2863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:19.220162 2863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:19.220691 2863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260925 05:19:19.220781 2863 leader_election.cc:302] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 877bec05db364b64b693ab12e35a5bfd; no voters:
I20260925 05:19:19.220912 2863 leader_election.cc:288] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:19.220980 2866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:19.221153 2866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [term 1 LEADER]: Becoming Leader. State: Replica: 877bec05db364b64b693ab12e35a5bfd, State: Running, Role: LEADER
I20260925 05:19:19.221191 2863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:19.221251 2866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260925 05:19:19.221771 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2795
I20260925 05:19:19.221791 2868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877bec05db364b64b693ab12e35a5bfd. Latest consensus state: current_term: 1 leader_uuid: "877bec05db364b64b693ab12e35a5bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } } }
I20260925 05:19:19.221869 2868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:19.221886 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:19.222088 13954 external_mini_cluster.cc:1428] Setting key 2c73f9558d2f386bb20c3f53cc3fddd9
I20260925 05:19:19.222262 2867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877bec05db364b64b693ab12e35a5bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bec05db364b64b693ab12e35a5bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } } }
I20260925 05:19:19.222736 2876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:19.223047 2867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:19.223544 2876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:19.225261 2876 catalog_manager.cc:1347] Generated new cluster ID: 8bd752762e63406c8e360a2e3e2deecd
I20260925 05:19:19.225314 2876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:19.228205 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:19.222783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:41425 (local address 127.0.0.1:35705)
0925 05:19:19.222984 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:19.222996 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:19.223040 (+ 44us) server_negotiation.cc:407] Connection header received
0925 05:19:19.223165 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:19.223172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:19.223234 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:19.223397 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:19.223652 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:19.224834 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:19.225982 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:19.226577 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:19.226681 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:19.227492 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:19.227497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:19.227497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:19.227724 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:19.227761 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:19.227776 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:19.227855 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:19.227971 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 05:19:19.227996 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260925 05:19:19.229194 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:19.233361 2876 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:19.233897 2876 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:19.234511 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:19.230873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:37477 (local address 127.0.0.1:35705)
0925 05:19:19.230950 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:19.230952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:19.231044 (+ 92us) server_negotiation.cc:407] Connection header received
0925 05:19:19.231101 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:19.231102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:19.231111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:19.231167 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:19.231316 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:19.232046 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:19.233104 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:19.233428 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:19.233471 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:19.234261 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:19.234262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:19.234262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:19.234266 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:19.234281 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:19.234286 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:19.234346 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:19.234429 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 05:19:19.234441 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 05:19:19.239733 2876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877bec05db364b64b693ab12e35a5bfd: Generated new TSK 0
I20260925 05:19:19.239930 2876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:19.242646 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2795
W20260925 05:19:19.246805 2779 connection.cc:535] client connection to 127.0.0.1:35705 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35705): Connection reset by peer (error 104)
2026-09-25T05:19:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:19 dist-test-slave-9tgz krb5kdc[2896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:19 dist-test-slave-9tgz krb5kdc[2896](info): set up 2 sockets
Sep 25 05:19:19 dist-test-slave-9tgz krb5kdc[2896](info): commencing operation
krb5kdc: starting...
W20260925 05:19:20.284039 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:20 dist-test-slave-9tgz krb5kdc[2896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 05:19:20.410322 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:20.491500 2909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:20.491748 2909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:20.491786 2909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:20.492950 2909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:20.493018 2909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:20.493042 2909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:20.493069 2909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:20.493101 2909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:20.494570 2909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:20.494860 2909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:20.495111 2909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:20.499871 2914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:20.499866 2917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:20.499889 2915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:20.500195 2909 server_base.cc:1034] running on GCE node
I20260925 05:19:20.500438 2909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:20.500722 2909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:20.501909 2909 hybrid_clock.cc:648] HybridClock initialized: now 1790313560501888 us; error 37 us; skew 500 ppm
Sep 25 05:19:20 dist-test-slave-9tgz krb5kdc[2896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:20.504336 2909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 05:19:20.504805 2909 webserver.cc:460] Webserver started at http://127.0.0.1:34247/ using document root <none> and password file <none>
I20260925 05:19:20.504987 2909 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:20.505026 2909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:20.505113 2909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:20.506155 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "55f2ae0a32814760834c562543d09f8a"
format_stamp: "Formatted at 2026-09-25 05:19:20 on dist-test-slave-9tgz"
server_key: "3179f77cceaa43353041a2cd0e471611"
server_key_iv: "38644db2e1c41774ed62ed4b84bdbc32"
server_key_version: "encryptionkey@0"
I20260925 05:19:20.506454 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "55f2ae0a32814760834c562543d09f8a"
format_stamp: "Formatted at 2026-09-25 05:19:20 on dist-test-slave-9tgz"
server_key: "3179f77cceaa43353041a2cd0e471611"
server_key_iv: "38644db2e1c41774ed62ed4b84bdbc32"
server_key_version: "encryptionkey@0"
I20260925 05:19:20.507826 2909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:20.508555 2924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:20.508786 2909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:20.508877 2909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "55f2ae0a32814760834c562543d09f8a"
format_stamp: "Formatted at 2026-09-25 05:19:20 on dist-test-slave-9tgz"
server_key: "3179f77cceaa43353041a2cd0e471611"
server_key_iv: "38644db2e1c41774ed62ed4b84bdbc32"
server_key_version: "encryptionkey@0"
I20260925 05:19:20.508934 2909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:20.512238 2909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:20.514529 2909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:20.514686 2909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:20.518568 2909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39977
I20260925 05:19:20.518625 2976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39977 every 8 connection(s)
I20260925 05:19:20.519032 2909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:20.519914 2977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:20.522161 2977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a: Bootstrap starting.
I20260925 05:19:20.522739 2977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:20.523096 2977 log.cc:824] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:20.523783 2977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a: No bootstrap required, opened a new log
I20260925 05:19:20.524897 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 2909
I20260925 05:19:20.525017 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:20.525192 13954 external_mini_cluster.cc:1428] Setting key 1b53dd56e480691f1a6b88e7246d3c3b
I20260925 05:19:20.525627 2977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } }
I20260925 05:19:20.525914 2977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:20.526001 2977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f2ae0a32814760834c562543d09f8a, State: Initialized, Role: FOLLOWER
I20260925 05:19:20.526147 2977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } }
I20260925 05:19:20.526257 2977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:20.526290 2977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:20.526396 2977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:20.527487 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:20.525718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:38975 (local address 127.0.0.1:39977)
0925 05:19:20.525918 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:20.525926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:20.525953 (+ 27us) server_negotiation.cc:407] Connection header received
0925 05:19:20.526111 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:20.526117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:20.526149 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:20.526286 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:20.527011 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:20.527014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:20.527014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:20.527024 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:20.527061 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:20.527072 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:20.527129 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:20.527225 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 05:19:20.527285 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260925 05:19:20.527654 2977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } }
I20260925 05:19:20.527761 2977 leader_election.cc:302] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f2ae0a32814760834c562543d09f8a; no voters:
I20260925 05:19:20.527928 2977 leader_election.cc:288] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:20.527956 2982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:20.528149 2982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [term 1 LEADER]: Becoming Leader. State: Replica: 55f2ae0a32814760834c562543d09f8a, State: Running, Role: LEADER
I20260925 05:19:20.528240 2977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:20.528280 2982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } }
I20260925 05:19:20.528368 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:20.528668 2982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55f2ae0a32814760834c562543d09f8a. Latest consensus state: current_term: 1 leader_uuid: "55f2ae0a32814760834c562543d09f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } } }
I20260925 05:19:20.528750 2982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:20.528896 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55f2ae0a32814760834c562543d09f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f2ae0a32814760834c562543d09f8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39977 } } }
I20260925 05:19:20.528970 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:20.529084 2989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:20.529626 2989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:20.531427 2989 catalog_manager.cc:1347] Generated new cluster ID: 89295bde41354e23964ac55a653c0ba9
I20260925 05:19:20.531481 2989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:20.531584 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:20.530202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.147:51555 (local address 127.0.0.1:39977)
0925 05:19:20.530233 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:20.530235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:20.530369 (+ 134us) server_negotiation.cc:407] Connection header received
0925 05:19:20.530458 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:20.530459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:20.530466 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:20.530494 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:20.531252 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:20.531254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:20.531254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:20.531258 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:20.531273 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:20.531278 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:20.531342 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:20.531438 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 05:19:20.531453 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 05:19:20.542843 2989 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:20.543079 2989 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:20.552557 2989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55f2ae0a32814760834c562543d09f8a: Generated new TSK 0
I20260925 05:19:20.552757 2989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:20.556409 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 2909
2026-09-25T05:19:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:20 dist-test-slave-9tgz krb5kdc[3008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:20 dist-test-slave-9tgz krb5kdc[3008](info): set up 2 sockets
Sep 25 05:19:20 dist-test-slave-9tgz krb5kdc[3008](info): commencing operation
krb5kdc: starting...
W20260925 05:19:21.580430 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:21.701116 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:42171
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:42171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:21.780771 3021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:21.781025 3021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:21.781064 3021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:21.782202 3021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:21.782274 3021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:21.782298 3021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:21.782328 3021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:21.782361 3021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:21.783844 3021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:42171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:42171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:21.784128 3021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:21.784370 3021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:19:21.789441 3021 server_base.cc:1034] running on GCE node
W20260925 05:19:21.789415 3029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:21.789393 3026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:21.789386 3027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:21.789888 3021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:21.790175 3021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:21.791335 3021 hybrid_clock.cc:648] HybridClock initialized: now 1790313561791324 us; error 44 us; skew 500 ppm
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313561, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:21.793834 3021 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:21.794298 3021 webserver.cc:460] Webserver started at http://127.13.160.190:37479/ using document root <none> and password file <none>
I20260925 05:19:21.794518 3021 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:21.794584 3021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:21.794734 3021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:21.795866 3021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "2f487d17920040a69bdfedc12f704560"
format_stamp: "Formatted at 2026-09-25 05:19:21 on dist-test-slave-9tgz"
server_key: "5a67d14afa94b4da50ff3d7eb040dcd8"
server_key_iv: "5ccd69387da03a2fe10e6998946482d6"
server_key_version: "encryptionkey@0"
I20260925 05:19:21.796231 3021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "2f487d17920040a69bdfedc12f704560"
format_stamp: "Formatted at 2026-09-25 05:19:21 on dist-test-slave-9tgz"
server_key: "5a67d14afa94b4da50ff3d7eb040dcd8"
server_key_iv: "5ccd69387da03a2fe10e6998946482d6"
server_key_version: "encryptionkey@0"
I20260925 05:19:21.797575 3021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:21.798327 3036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:21.798518 3021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:21.798614 3021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2f487d17920040a69bdfedc12f704560"
format_stamp: "Formatted at 2026-09-25 05:19:21 on dist-test-slave-9tgz"
server_key: "5a67d14afa94b4da50ff3d7eb040dcd8"
server_key_iv: "5ccd69387da03a2fe10e6998946482d6"
server_key_version: "encryptionkey@0"
I20260925 05:19:21.798697 3021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:21.811571 3021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:21.813448 3021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:21.813611 3021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:21.817711 3021 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:42171
I20260925 05:19:21.817746 3089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:42171 every 8 connection(s)
I20260925 05:19:21.818133 3021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:21.819087 3090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:21.821838 3090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560: Bootstrap starting.
I20260925 05:19:21.822365 3090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:21.822606 3090 log.cc:824] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:21.823282 3090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560: No bootstrap required, opened a new log
I20260925 05:19:21.824842 3090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } }
I20260925 05:19:21.824985 3090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:21.825007 3090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f487d17920040a69bdfedc12f704560, State: Initialized, Role: FOLLOWER
I20260925 05:19:21.825152 3090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } }
I20260925 05:19:21.825235 3090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:21.825258 3090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:21.825287 3090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:21.825685 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3021
I20260925 05:19:21.825834 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:21.826067 13954 external_mini_cluster.cc:1428] Setting key 704dfb60d0be9ef07ad517549a6af6f2
I20260925 05:19:21.826442 3090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } }
I20260925 05:19:21.826592 3090 leader_election.cc:302] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f487d17920040a69bdfedc12f704560; no voters:
I20260925 05:19:21.826776 3090 leader_election.cc:288] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:21.826840 3095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:21.827011 3095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [term 1 LEADER]: Becoming Leader. State: Replica: 2f487d17920040a69bdfedc12f704560, State: Running, Role: LEADER
I20260925 05:19:21.827078 3090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:21.827206 3095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } }
I20260925 05:19:21.827643 3096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f487d17920040a69bdfedc12f704560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } } }
I20260925 05:19:21.827733 3096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:21.827939 3097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f487d17920040a69bdfedc12f704560. Latest consensus state: current_term: 1 leader_uuid: "2f487d17920040a69bdfedc12f704560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f487d17920040a69bdfedc12f704560" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 42171 } } }
I20260925 05:19:21.828052 3097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:21.828370 3104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:21.828909 3104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:21.830314 3104 catalog_manager.cc:1347] Generated new cluster ID: 5cfbb04e43574a2f917ae1a3b66b9ac5
I20260925 05:19:21.830366 3104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:21.838706 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:21.826601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48355 (local address 127.13.160.190:42171)
0925 05:19:21.826816 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:21.826845 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:21.826880 (+ 35us) server_negotiation.cc:407] Connection header received
0925 05:19:21.827027 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:21.827034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:21.827093 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:21.827998 (+ 905us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:21.828116 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:21.829654 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:21.830672 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:21.831284 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:21.831386 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:21.834233 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:21.834238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:21.834239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:21.834255 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:21.836886 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:21.837338 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:21.837342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:21.837342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:21.837469 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:21.837714 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:21.837715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:21.837715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:21.838064 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:21.838229 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:21.838347 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 05:19:21.838453 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260925 05:19:21.839812 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:21.843004 3104 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:21.843665 3104 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:21.848572 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:21.841333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49017 (local address 127.13.160.190:42171)
0925 05:19:21.841368 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:21.841370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:21.841478 (+ 108us) server_negotiation.cc:407] Connection header received
0925 05:19:21.841612 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:21.841613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:21.841625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:21.841684 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:21.842298 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:21.843098 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:21.844009 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:21.844341 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:21.844412 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:21.845743 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:21.845745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:21.845745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:21.845764 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:21.847392 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:21.847784 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:21.847785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:21.847785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:21.847848 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:21.848069 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:21.848070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:21.848071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:21.848198 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:21.848312 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:21.848416 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 05:19:21.848480 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:19:21.857367 3104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f487d17920040a69bdfedc12f704560: Generated new TSK 0
I20260925 05:19:21.857599 3104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:21.859719 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3021
W20260925 05:19:21.864686 3006 connection.cc:535] client connection to 127.13.160.190:42171 recv error: Network error: failed to read from TLS socket (remote: 127.13.160.190:42171): Connection reset by peer (error 104)
2026-09-25T05:19:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3121](info): set up 2 sockets
Sep 25 05:19:21 dist-test-slave-9tgz krb5kdc[3121](info): commencing operation
krb5kdc: starting...
W20260925 05:19:22.892169 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:22 dist-test-slave-9tgz krb5kdc[3121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:23.013975 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37525
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:37525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:23.094698 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:23.094933 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:23.094959 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:23.096117 3134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:23.096166 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:23.096181 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:23.096194 3134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:23.096246 3134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:23.097704 3134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:37525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:23.097963 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:23.098218 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:23.103081 3139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:23.103082 3142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:23.103137 3134 server_base.cc:1034] running on GCE node
W20260925 05:19:23.103097 3140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:23.103518 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:23.103866 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:23.105031 3134 hybrid_clock.cc:648] HybridClock initialized: now 1790313563105021 us; error 47 us; skew 500 ppm
Sep 25 05:19:23 dist-test-slave-9tgz krb5kdc[3121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313563, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:23.107474 3134 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:23.107931 3134 webserver.cc:460] Webserver started at http://127.13.160.190:33001/ using document root <none> and password file <none>
I20260925 05:19:23.108127 3134 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:23.108187 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:23.108327 3134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:23.109387 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "3879b0f26ecf47edb57dc0340fe53d2c"
format_stamp: "Formatted at 2026-09-25 05:19:23 on dist-test-slave-9tgz"
server_key: "07205782743bb78d840acfc32aa9c929"
server_key_iv: "ca79a974184b72e97b51aa30b26eef2f"
server_key_version: "encryptionkey@0"
I20260925 05:19:23.109747 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "3879b0f26ecf47edb57dc0340fe53d2c"
format_stamp: "Formatted at 2026-09-25 05:19:23 on dist-test-slave-9tgz"
server_key: "07205782743bb78d840acfc32aa9c929"
server_key_iv: "ca79a974184b72e97b51aa30b26eef2f"
server_key_version: "encryptionkey@0"
I20260925 05:19:23.111084 3134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 05:19:23.111790 3149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:23.111965 3134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:23.112056 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "3879b0f26ecf47edb57dc0340fe53d2c"
format_stamp: "Formatted at 2026-09-25 05:19:23 on dist-test-slave-9tgz"
server_key: "07205782743bb78d840acfc32aa9c929"
server_key_iv: "ca79a974184b72e97b51aa30b26eef2f"
server_key_version: "encryptionkey@0"
I20260925 05:19:23.112133 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:23.129606 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:23.132025 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:23.132174 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:23.136348 3134 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:37525
I20260925 05:19:23.136413 3201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:37525 every 8 connection(s)
I20260925 05:19:23.136749 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:23.137620 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:23.138813 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3134
I20260925 05:19:23.138967 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:23.139194 13954 external_mini_cluster.cc:1428] Setting key 2d0a7da85e119da7ae20e5e90083e303
I20260925 05:19:23.140686 3202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c: Bootstrap starting.
I20260925 05:19:23.141320 3202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:23.141630 3202 log.cc:824] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:23.142479 3202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c: No bootstrap required, opened a new log
I20260925 05:19:23.144964 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:23.139766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48253 (local address 127.13.160.190:37525)
0925 05:19:23.139943 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:23.139951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:23.139976 (+ 25us) server_negotiation.cc:407] Connection header received
0925 05:19:23.140418 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:23.140425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:23.140472 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:23.140603 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:23.140886 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:23.141915 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:23.143020 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:23.143425 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:23.143511 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:23.144285 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:23.144289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:23.144289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:23.144498 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:23.144528 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:23.144539 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:23.144618 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:23.144781 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 05:19:23.144809 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260925 05:19:23.145010 3202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } }
I20260925 05:19:23.145211 3202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:23.145327 3202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3879b0f26ecf47edb57dc0340fe53d2c, State: Initialized, Role: FOLLOWER
I20260925 05:19:23.145493 3202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } }
I20260925 05:19:23.145613 3202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:23.145650 3202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:23.145694 3202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:23.145996 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:23.146425 3202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } }
I20260925 05:19:23.146564 3202 leader_election.cc:302] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3879b0f26ecf47edb57dc0340fe53d2c; no voters:
I20260925 05:19:23.146786 3202 leader_election.cc:288] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:23.146809 3207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:23.146952 3207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [term 1 LEADER]: Becoming Leader. State: Replica: 3879b0f26ecf47edb57dc0340fe53d2c, State: Running, Role: LEADER
I20260925 05:19:23.147115 3207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } }
I20260925 05:19:23.147089 3202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:23.147542 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } } }
I20260925 05:19:23.147588 3209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3879b0f26ecf47edb57dc0340fe53d2c. Latest consensus state: current_term: 1 leader_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3879b0f26ecf47edb57dc0340fe53d2c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37525 } } }
I20260925 05:19:23.147703 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:23.147763 3209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:23.148051 3217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:23.148752 3217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:23.150393 3217 catalog_manager.cc:1347] Generated new cluster ID: c784900cf9f1438e9f4b5a0fd171081c
I20260925 05:19:23.150483 3217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:23.151423 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:23.147530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60227 (local address 127.13.160.190:37525)
0925 05:19:23.147568 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:23.147570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:23.147848 (+ 278us) server_negotiation.cc:407] Connection header received
0925 05:19:23.147953 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:23.147954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:23.147968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:23.148022 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:23.148278 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:23.148976 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:23.150052 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:23.150285 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:23.150346 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:23.151126 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:23.151127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:23.151128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:23.151133 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:23.151150 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:23.151155 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:23.151234 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:23.151333 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 05:19:23.151348 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:19:23.156697 3217 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:23.157191 3217 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:23.168522 3217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3879b0f26ecf47edb57dc0340fe53d2c: Generated new TSK 0
I20260925 05:19:23.168712 3217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:23.176234 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3134
W20260925 05:19:23.180536 3119 connection.cc:535] client connection to 127.13.160.190:37525 recv error: Network error: failed to read from TLS socket (remote: 127.13.160.190:37525): Connection reset by peer (error 104)
2026-09-25T05:19:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:23 dist-test-slave-9tgz krb5kdc[3234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:23 dist-test-slave-9tgz krb5kdc[3234](info): set up 2 sockets
Sep 25 05:19:23 dist-test-slave-9tgz krb5kdc[3234](info): commencing operation
krb5kdc: starting...
W20260925 05:19:24.219924 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:24 dist-test-slave-9tgz krb5kdc[3234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:24.337915 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37733
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:37733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:24.419116 3247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:24.419339 3247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:24.419363 3247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:24.420555 3247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:24.420599 3247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:24.420614 3247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:24.420627 3247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:24.420684 3247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:24.422158 3247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:37733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:24.422421 3247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:24.422655 3247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:24.427525 3253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:24.427556 3252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:24.427546 3255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:24.427584 3247 server_base.cc:1034] running on GCE node
I20260925 05:19:24.427958 3247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:24.428246 3247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:24.429387 3247 hybrid_clock.cc:648] HybridClock initialized: now 1790313564429377 us; error 28 us; skew 500 ppm
Sep 25 05:19:24 dist-test-slave-9tgz krb5kdc[3234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313564, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:24.435563 3247 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:24.436051 3247 webserver.cc:460] Webserver started at http://127.13.160.190:46007/ using document root <none> and password file <none>
I20260925 05:19:24.436295 3247 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:24.436364 3247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:24.436518 3247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:24.437570 3247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "568091130a33486c8d4ea0790208bf6e"
format_stamp: "Formatted at 2026-09-25 05:19:24 on dist-test-slave-9tgz"
server_key: "f402ac862a0371400b0fce84e54532d1"
server_key_iv: "7cd6fe8a815d6ab36934baad71e00ada"
server_key_version: "encryptionkey@0"
I20260925 05:19:24.437932 3247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "568091130a33486c8d4ea0790208bf6e"
format_stamp: "Formatted at 2026-09-25 05:19:24 on dist-test-slave-9tgz"
server_key: "f402ac862a0371400b0fce84e54532d1"
server_key_iv: "7cd6fe8a815d6ab36934baad71e00ada"
server_key_version: "encryptionkey@0"
I20260925 05:19:24.439328 3247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:24.440151 3262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:24.440464 3247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:24.440569 3247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "568091130a33486c8d4ea0790208bf6e"
format_stamp: "Formatted at 2026-09-25 05:19:24 on dist-test-slave-9tgz"
server_key: "f402ac862a0371400b0fce84e54532d1"
server_key_iv: "7cd6fe8a815d6ab36934baad71e00ada"
server_key_version: "encryptionkey@0"
I20260925 05:19:24.440644 3247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:24.443981 3247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:24.445847 3247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:24.446008 3247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:24.449915 3247 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:37733
I20260925 05:19:24.449949 3315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:37733 every 8 connection(s)
I20260925 05:19:24.450287 3247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:24.451315 3316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:24.452531 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3247
I20260925 05:19:24.452675 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:24.452873 13954 external_mini_cluster.cc:1428] Setting key de2886ac00295b6a2125e4aecf6f18fb
I20260925 05:19:24.454022 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e: Bootstrap starting.
I20260925 05:19:24.454726 3316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:24.455106 3316 log.cc:824] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:24.455535 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:24.453412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38321 (local address 127.13.160.190:37733)
0925 05:19:24.453618 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:24.453628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:24.453664 (+ 36us) server_negotiation.cc:407] Connection header received
0925 05:19:24.454200 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:24.454210 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:24.454244 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:24.454408 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:24.455087 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:24.455090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:24.455090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:24.455094 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:24.455128 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:24.455139 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:24.455245 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:24.455308 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 05:19:24.455351 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260925 05:19:24.455878 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e: No bootstrap required, opened a new log
I20260925 05:19:24.456624 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:24.458439 3316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } }
I20260925 05:19:24.458600 3316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:24.458645 3316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 568091130a33486c8d4ea0790208bf6e, State: Initialized, Role: FOLLOWER
I20260925 05:19:24.458772 3316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } }
I20260925 05:19:24.458881 3316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:24.458958 3316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:24.459012 3316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:24.459407 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:24.458250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45791 (local address 127.13.160.190:37733)
0925 05:19:24.458305 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:24.458307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:24.458426 (+ 119us) server_negotiation.cc:407] Connection header received
0925 05:19:24.458499 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:24.458500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:24.458505 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:24.458567 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:24.459195 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:24.459196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:24.459196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:24.459200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:24.459213 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:24.459217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:24.459273 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:24.459329 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 05:19:24.459343 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 05:19:24.459686 3316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } }
I20260925 05:19:24.459811 3316 leader_election.cc:302] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 568091130a33486c8d4ea0790208bf6e; no voters:
I20260925 05:19:24.459981 3316 leader_election.cc:288] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:24.460016 3326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:24.460175 3326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [term 1 LEADER]: Becoming Leader. State: Replica: 568091130a33486c8d4ea0790208bf6e, State: Running, Role: LEADER
I20260925 05:19:24.460247 3316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:24.460304 3326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } }
I20260925 05:19:24.460695 3328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 568091130a33486c8d4ea0790208bf6e. Latest consensus state: current_term: 1 leader_uuid: "568091130a33486c8d4ea0790208bf6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } } }
I20260925 05:19:24.460801 3328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:24.461021 3327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "568091130a33486c8d4ea0790208bf6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "568091130a33486c8d4ea0790208bf6e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37733 } } }
I20260925 05:19:24.461128 3327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:24.461438 3334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:24.461966 3334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:24.463431 3334 catalog_manager.cc:1347] Generated new cluster ID: 3b1ce096fc384e208c32f2bb221760c9
I20260925 05:19:24.463492 3334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:24.489398 3334 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:24.489615 3334 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:24.492204 3334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 568091130a33486c8d4ea0790208bf6e: Generated new TSK 0
I20260925 05:19:24.492405 3334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:24.497341 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3247
2026-09-25T05:19:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:24 dist-test-slave-9tgz krb5kdc[3347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:24 dist-test-slave-9tgz krb5kdc[3347](info): set up 2 sockets
Sep 25 05:19:24 dist-test-slave-9tgz krb5kdc[3347](info): commencing operation
krb5kdc: starting...
W20260925 05:19:25.532033 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:25.657914 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36631
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:36631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:25.742046 3360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:25.742362 3360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:25.742408 3360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:25.743747 3360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:25.743845 3360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:25.743871 3360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:25.743891 3360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:25.743921 3360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:25.745599 3360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:36631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:36631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:25.745954 3360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:25.746217 3360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:25.751343 3365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:25.751372 3366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:25.751343 3368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:25.751650 3360 server_base.cc:1034] running on GCE node
I20260925 05:19:25.751910 3360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:25.752280 3360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:25.753450 3360 hybrid_clock.cc:648] HybridClock initialized: now 1790313565753443 us; error 40 us; skew 500 ppm
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313565, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:25.756338 3360 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:25.756834 3360 webserver.cc:460] Webserver started at http://127.13.160.190:38349/ using document root <none> and password file <none>
I20260925 05:19:25.757076 3360 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:25.757156 3360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:25.757313 3360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:25.758461 3360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "30aca472177c481ba70960cf5ff5bee4"
format_stamp: "Formatted at 2026-09-25 05:19:25 on dist-test-slave-9tgz"
server_key: "562e1108e34e458279a0c0dde5802fcd"
server_key_iv: "5452b7162678a3208b10c6bb569719ac"
server_key_version: "encryptionkey@0"
I20260925 05:19:25.758852 3360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "30aca472177c481ba70960cf5ff5bee4"
format_stamp: "Formatted at 2026-09-25 05:19:25 on dist-test-slave-9tgz"
server_key: "562e1108e34e458279a0c0dde5802fcd"
server_key_iv: "5452b7162678a3208b10c6bb569719ac"
server_key_version: "encryptionkey@0"
I20260925 05:19:25.760330 3360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:19:25.761137 3375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:25.761334 3360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 05:19:25.761447 3360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "30aca472177c481ba70960cf5ff5bee4"
format_stamp: "Formatted at 2026-09-25 05:19:25 on dist-test-slave-9tgz"
server_key: "562e1108e34e458279a0c0dde5802fcd"
server_key_iv: "5452b7162678a3208b10c6bb569719ac"
server_key_version: "encryptionkey@0"
I20260925 05:19:25.761529 3360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:25.776075 3360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:25.778788 3360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:25.779002 3360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:25.783277 3360 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:36631
I20260925 05:19:25.783330 3427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:36631 every 8 connection(s)
I20260925 05:19:25.783738 3360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:25.784732 3428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:25.787367 3428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: Bootstrap starting.
I20260925 05:19:25.788015 3428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:25.788326 3428 log.cc:824] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:25.789139 3428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: No bootstrap required, opened a new log
I20260925 05:19:25.790856 3428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } }
I20260925 05:19:25.791112 3428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:25.791168 3428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30aca472177c481ba70960cf5ff5bee4, State: Initialized, Role: FOLLOWER
I20260925 05:19:25.791332 3428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } }
I20260925 05:19:25.791399 3428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:25.791422 3428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:25.791491 3428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:25.792476 3428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } }
I20260925 05:19:25.792601 3428 leader_election.cc:302] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30aca472177c481ba70960cf5ff5bee4; no voters:
I20260925 05:19:25.792842 3428 leader_election.cc:288] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:25.792893 3431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:25.793112 3431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [term 1 LEADER]: Becoming Leader. State: Replica: 30aca472177c481ba70960cf5ff5bee4, State: Running, Role: LEADER
I20260925 05:19:25.793155 3428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:25.793215 3431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } }
I20260925 05:19:25.793385 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3360
I20260925 05:19:25.793478 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:25.793671 13954 external_mini_cluster.cc:1428] Setting key 7c043b22c9646fa8538aeaf7cfaa05e7
I20260925 05:19:25.794008 3433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30aca472177c481ba70960cf5ff5bee4. Latest consensus state: current_term: 1 leader_uuid: "30aca472177c481ba70960cf5ff5bee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } } }
I20260925 05:19:25.794098 3433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:25.794559 3432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30aca472177c481ba70960cf5ff5bee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aca472177c481ba70960cf5ff5bee4" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 36631 } } }
I20260925 05:19:25.794988 3432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4 [sys.catalog]: This master's current role is: LEADER
W20260925 05:19:25.795226 3449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:25.795288 3449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:25.795344 3441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:25.795539 3441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:25.797217 3441 catalog_manager.cc:1347] Generated new cluster ID: 478d8b3fa239433495eb52747c1517b4
I20260925 05:19:25.797293 3441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:25.805414 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:25.794388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60857 (local address 127.13.160.190:36631)
0925 05:19:25.794572 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:25.794583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:25.794636 (+ 53us) server_negotiation.cc:407] Connection header received
0925 05:19:25.794778 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:25.794787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:25.794857 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:25.795907 (+ 1050us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:25.795992 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:25.797263 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:25.798115 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:25.798579 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:25.798716 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:25.801319 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:25.801325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:25.801325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:25.801340 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:25.803521 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:25.804003 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:25.804007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:25.804007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:25.804141 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:25.804378 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:25.804379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:25.804379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:25.804715 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:25.804875 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:25.805021 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 05:19:25.805171 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260925 05:19:25.806946 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:25.808547 3441 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:25.809342 3441 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:25.815272 3441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30aca472177c481ba70960cf5ff5bee4: Generated new TSK 0
I20260925 05:19:25.815536 3441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:25.817214 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:25.808577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45653 (local address 127.13.160.190:36631)
0925 05:19:25.808619 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:25.808622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:25.808873 (+ 251us) server_negotiation.cc:407] Connection header received
0925 05:19:25.808979 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:25.808980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:25.808990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:25.809019 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:25.809791 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:25.810543 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:25.811623 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:25.811935 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:25.811974 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:25.813598 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:25.813600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:25.813600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:25.813609 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:25.815719 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:25.816213 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:25.816214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:25.816215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:25.816324 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:25.816559 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:25.816561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:25.816561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:25.816697 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:25.816820 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:25.816966 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 05:19:25.817100 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":20}
I20260925 05:19:25.819412 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3360
2026-09-25T05:19:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3459](info): set up 2 sockets
Sep 25 05:19:25 dist-test-slave-9tgz krb5kdc[3459](info): commencing operation
krb5kdc: starting...
W20260925 05:19:26.847967 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:26 dist-test-slave-9tgz krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:26.969638 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37903
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:27.055951 3472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:27.056191 3472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:27.056216 3472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:27.057471 3472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:27.057526 3472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:27.057541 3472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:27.057555 3472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:27.057567 3472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:27.059227 3472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:37903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:27.059544 3472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:27.059823 3472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:27.064967 3480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:27.064967 3477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:27.065058 3478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:27.065461 3472 server_base.cc:1034] running on GCE node
I20260925 05:19:27.065707 3472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:27.065987 3472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:27.067178 3472 hybrid_clock.cc:648] HybridClock initialized: now 1790313567067167 us; error 43 us; skew 500 ppm
Sep 25 05:19:27 dist-test-slave-9tgz krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313567, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:27.069681 3472 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:27.070147 3472 webserver.cc:460] Webserver started at http://127.13.160.190:43295/ using document root <none> and password file <none>
I20260925 05:19:27.070398 3472 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:27.070472 3472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:27.070623 3472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:27.071790 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "5186fb67a82b4dc7bc5ab82d696b994f"
format_stamp: "Formatted at 2026-09-25 05:19:27 on dist-test-slave-9tgz"
server_key: "f4446521790dbf1477bcf814c995e3ae"
server_key_iv: "965306d696bbec8e9dfc4c61a62d317f"
server_key_version: "encryptionkey@0"
I20260925 05:19:27.072160 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "5186fb67a82b4dc7bc5ab82d696b994f"
format_stamp: "Formatted at 2026-09-25 05:19:27 on dist-test-slave-9tgz"
server_key: "f4446521790dbf1477bcf814c995e3ae"
server_key_iv: "965306d696bbec8e9dfc4c61a62d317f"
server_key_version: "encryptionkey@0"
I20260925 05:19:27.073510 3472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:19:27.074234 3487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:27.074438 3472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 05:19:27.074572 3472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "5186fb67a82b4dc7bc5ab82d696b994f"
format_stamp: "Formatted at 2026-09-25 05:19:27 on dist-test-slave-9tgz"
server_key: "f4446521790dbf1477bcf814c995e3ae"
server_key_iv: "965306d696bbec8e9dfc4c61a62d317f"
server_key_version: "encryptionkey@0"
I20260925 05:19:27.074679 3472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:27.088726 3472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:27.090571 3472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:27.090694 3472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:27.094539 3472 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:37903
I20260925 05:19:27.094621 3540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:37903 every 8 connection(s)
I20260925 05:19:27.095072 3472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:27.096050 3541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:27.098413 3541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f: Bootstrap starting.
I20260925 05:19:27.099052 3541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:27.099356 3541 log.cc:824] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:27.100081 3541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f: No bootstrap required, opened a new log
I20260925 05:19:27.101728 3541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } }
I20260925 05:19:27.101903 3541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:27.101950 3541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5186fb67a82b4dc7bc5ab82d696b994f, State: Initialized, Role: FOLLOWER
I20260925 05:19:27.102130 3541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } }
I20260925 05:19:27.102217 3541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:27.102262 3541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:27.102314 3541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:27.103262 3541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } }
I20260925 05:19:27.103392 3541 leader_election.cc:302] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5186fb67a82b4dc7bc5ab82d696b994f; no voters:
I20260925 05:19:27.103590 3541 leader_election.cc:288] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:27.103639 3544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:27.103869 3544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [term 1 LEADER]: Becoming Leader. State: Replica: 5186fb67a82b4dc7bc5ab82d696b994f, State: Running, Role: LEADER
I20260925 05:19:27.103878 3541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:27.103973 3544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } }
I20260925 05:19:27.104454 3545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } } }
I20260925 05:19:27.104602 3546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5186fb67a82b4dc7bc5ab82d696b994f. Latest consensus state: current_term: 1 leader_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186fb67a82b4dc7bc5ab82d696b994f" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 37903 } } }
I20260925 05:19:27.104662 3545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:27.104724 3546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:27.105206 3553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:27.105213 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3472
I20260925 05:19:27.105326 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:27.105515 13954 external_mini_cluster.cc:1428] Setting key de6e4f0b5327953e5d96d23ee3bfc984
I20260925 05:19:27.105912 3553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:27.108340 3553 catalog_manager.cc:1347] Generated new cluster ID: e5991020e6e840e9a787a05ea7316a15
I20260925 05:19:27.108404 3553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:27.111617 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:27.106178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35109 (local address 127.13.160.190:37903)
0925 05:19:27.106386 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:27.106395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:27.106429 (+ 34us) server_negotiation.cc:407] Connection header received
0925 05:19:27.106557 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:27.106565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:27.106625 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:27.106791 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:27.107058 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:27.108171 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:27.109303 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:27.109968 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:27.110095 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:27.110843 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:27.110848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:27.110848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:27.111104 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:27.111147 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:27.111163 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:27.111249 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:27.111324 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 05:19:27.111398 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260925 05:19:27.113026 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:27.118718 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:27.114508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52505 (local address 127.13.160.190:37903)
0925 05:19:27.114581 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:27.114583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:27.115090 (+ 507us) server_negotiation.cc:407] Connection header received
0925 05:19:27.115197 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:27.115198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:27.115209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:27.115240 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:27.115448 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:27.116178 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:27.117272 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:27.117586 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:27.117628 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:27.118443 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:27.118445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:27.118445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:27.118451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:27.118465 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:27.118470 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:27.118499 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:27.118603 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 05:19:27.118633 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 05:19:27.132616 3553 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:27.133219 3553 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:27.140981 3553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5186fb67a82b4dc7bc5ab82d696b994f: Generated new TSK 0
I20260925 05:19:27.141216 3553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:27.146101 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3472
2026-09-25T05:19:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:27 dist-test-slave-9tgz krb5kdc[3572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:27 dist-test-slave-9tgz krb5kdc[3572](info): set up 2 sockets
Sep 25 05:19:27 dist-test-slave-9tgz krb5kdc[3572](info): commencing operation
krb5kdc: starting...
W20260925 05:19:28.188154 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:28 dist-test-slave-9tgz krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:28.310078 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33973
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:28.393296 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:28.393540 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:28.393582 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:28.394778 3585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:28.394847 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:28.394874 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:28.394930 3585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:28.394984 3585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:28.396477 3585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:28.396761 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:28.396992 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:28.402016 3590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:28.402205 3585 server_base.cc:1034] running on GCE node
W20260925 05:19:28.402019 3591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:28.402019 3593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:28.402520 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:28.402853 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:28.404054 3585 hybrid_clock.cc:648] HybridClock initialized: now 1790313568404038 us; error 30 us; skew 500 ppm
Sep 25 05:19:28 dist-test-slave-9tgz krb5kdc[3572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313568, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:28.406697 3585 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:28.407264 3585 webserver.cc:460] Webserver started at http://127.13.160.190:42405/ using document root <none> and password file <none>
I20260925 05:19:28.407465 3585 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:28.407508 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:28.407603 3585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:28.408702 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "15bbf31805ee4638bc3668505894f03e"
format_stamp: "Formatted at 2026-09-25 05:19:28 on dist-test-slave-9tgz"
server_key: "9e6c3c77a7de82e4534d7121d8a4ac23"
server_key_iv: "574bd4dd49d6653191ec679f3d6ca341"
server_key_version: "encryptionkey@0"
I20260925 05:19:28.409020 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "15bbf31805ee4638bc3668505894f03e"
format_stamp: "Formatted at 2026-09-25 05:19:28 on dist-test-slave-9tgz"
server_key: "9e6c3c77a7de82e4534d7121d8a4ac23"
server_key_iv: "574bd4dd49d6653191ec679f3d6ca341"
server_key_version: "encryptionkey@0"
I20260925 05:19:28.410415 3585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:19:28.411226 3600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:28.411432 3585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 05:19:28.411512 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "15bbf31805ee4638bc3668505894f03e"
format_stamp: "Formatted at 2026-09-25 05:19:28 on dist-test-slave-9tgz"
server_key: "9e6c3c77a7de82e4534d7121d8a4ac23"
server_key_iv: "574bd4dd49d6653191ec679f3d6ca341"
server_key_version: "encryptionkey@0"
I20260925 05:19:28.411576 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:28.415292 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:28.417881 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:28.418037 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:28.422300 3585 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:33973
I20260925 05:19:28.422334 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:33973 every 8 connection(s)
I20260925 05:19:28.422700 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:28.423782 3654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:28.424993 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3585
I20260925 05:19:28.425096 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:28.425305 13954 external_mini_cluster.cc:1428] Setting key b446165d8df4a8ce79675b0bf28e8609
I20260925 05:19:28.426759 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e: Bootstrap starting.
I20260925 05:19:28.427525 3654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:28.427850 3654 log.cc:824] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:28.428133 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:28.425924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36871 (local address 127.13.160.190:33973)
0925 05:19:28.426131 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:28.426141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:28.426178 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:19:28.426732 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:28.426742 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:28.426782 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:28.426964 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:28.427652 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:28.427656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:28.427656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:28.427661 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:28.427712 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:28.427724 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:28.427813 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:28.427882 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 05:19:28.427930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260925 05:19:28.428716 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e: No bootstrap required, opened a new log
I20260925 05:19:28.429159 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:28.431644 3654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } }
I20260925 05:19:28.431860 3654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 05:19:28.431908 3654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15bbf31805ee4638bc3668505894f03e, State: Initialized, Role: FOLLOWER
I20260925 05:19:28.432061 3654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } }
I20260925 05:19:28.432137 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:28.430803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44555 (local address 127.13.160.190:33973)
0925 05:19:28.430869 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:28.430871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:28.431025 (+ 154us) server_negotiation.cc:407] Connection header received
0925 05:19:28.431096 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:28.431097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:28.431118 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:28.431164 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:28.431798 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:28.431799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:28.431800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 05:19:28.431803 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:28.431817 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 05:19:28.431821 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:28.431873 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:28.432065 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 05:19:28.432079 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:19:28.432189 3654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 05:19:28.432229 3654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 05:19:28.432276 3654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:28.433022 3654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } }
I20260925 05:19:28.433116 3654 leader_election.cc:302] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15bbf31805ee4638bc3668505894f03e; no voters:
I20260925 05:19:28.433270 3654 leader_election.cc:288] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 05:19:28.433379 3664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:28.433517 3654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:28.433537 3664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [term 1 LEADER]: Becoming Leader. State: Replica: 15bbf31805ee4638bc3668505894f03e, State: Running, Role: LEADER
I20260925 05:19:28.433645 3664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } }
I20260925 05:19:28.434041 3665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15bbf31805ee4638bc3668505894f03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } } }
I20260925 05:19:28.434062 3666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15bbf31805ee4638bc3668505894f03e. Latest consensus state: current_term: 1 leader_uuid: "15bbf31805ee4638bc3668505894f03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15bbf31805ee4638bc3668505894f03e" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 33973 } } }
I20260925 05:19:28.434198 3666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:28.434415 3665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:28.434439 3673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:28.435338 3673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:28.437026 3673 catalog_manager.cc:1347] Generated new cluster ID: 7da33e07456f4ab98663d6f35057b753
I20260925 05:19:28.437093 3673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:28.455042 3673 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:28.455281 3673 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:28.459218 3673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15bbf31805ee4638bc3668505894f03e: Generated new TSK 0
I20260925 05:19:28.459443 3673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 05:19:28.467736 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3585
2026-09-25T05:19:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1321 ms)
[----------] 18 tests from AuthTokenIssuingTest (23946 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:28 dist-test-slave-9tgz krb5kdc[3685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:28 dist-test-slave-9tgz krb5kdc[3685](info): set up 2 sockets
Sep 25 05:19:28 dist-test-slave-9tgz krb5kdc[3685](info): commencing operation
krb5kdc: starting...
W20260925 05:19:29.507869 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:29.638134 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41957
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:29.722054 3698 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:29.722306 3698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:29.722324 3698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:29.722347 3698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:29.723694 3698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:29.723749 3698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:29.723764 3698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:29.723778 3698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:29.723789 3698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:29.725288 3698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:29.725505 3698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:29.725708 3698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:29.730677 3703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:29.730713 3704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:29.730707 3706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:29.730913 3698 server_base.cc:1034] running on GCE node
I20260925 05:19:29.731174 3698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:29.731494 3698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:29.732649 3698 hybrid_clock.cc:648] HybridClock initialized: now 1790313569732639 us; error 28 us; skew 500 ppm
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:29.740157 3698 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:29.740689 3698 webserver.cc:460] Webserver started at http://127.13.160.190:41573/ using document root <none> and password file <none>
I20260925 05:19:29.740923 3698 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:29.740993 3698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:29.741156 3698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:29.742287 3698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "2658b42dd79f4ba8b273ab74d92eb10c"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "b503c9162d089dfa0758f4b2e984f9fc"
server_key_iv: "def645fa7abd3c8ec68fd9c232d73c16"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.742666 3698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "2658b42dd79f4ba8b273ab74d92eb10c"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "b503c9162d089dfa0758f4b2e984f9fc"
server_key_iv: "def645fa7abd3c8ec68fd9c232d73c16"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.744239 3698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:29.745002 3713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:29.745183 3698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:29.745281 3698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "2658b42dd79f4ba8b273ab74d92eb10c"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "b503c9162d089dfa0758f4b2e984f9fc"
server_key_iv: "def645fa7abd3c8ec68fd9c232d73c16"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.745368 3698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:29.755628 3698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:29.758141 3698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:29.758319 3698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:29.762521 3698 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:41957
I20260925 05:19:29.762555 3766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:41957 every 8 connection(s)
I20260925 05:19:29.763008 3698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:29.763247 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3698
I20260925 05:19:29.763376 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:29.763545 13954 external_mini_cluster.cc:1428] Setting key 9f29e33c0722b7d02d72de98c3aed3d6
I20260925 05:19:29.764031 3767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:29.765631 3767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.160.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.189@KRBTEST.COM" created.
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:29.778389 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.766502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59386 (local address 127.13.160.190:41957)
0925 05:19:29.766681 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:29.766688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:29.766795 (+ 107us) server_negotiation.cc:407] Connection header received
0925 05:19:29.767489 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:29.767491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:29.767512 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:29.767628 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:29.768223 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.769079 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.769978 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.770307 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.770439 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.774792 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:29.774802 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:29.774802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:29.774810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:29.776602 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.777140 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.777143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.777143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.777232 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.777517 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.777517 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.777518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.777761 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:29.777827 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:29.778082 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 05:19:29.778188 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260925 05:19:29.778515 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.766444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41957 (local address 127.0.0.1:59386)
0925 05:19:29.766626 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.766678 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:29.767317 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:29.767634 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:29.767637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:29.767948 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:29.768113 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.768117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.769193 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.769194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.769879 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.769881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.770409 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.770410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.770473 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.771087 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:29.771096 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:29.774654 (+ 3558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:29.776713 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.776722 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.776725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.777050 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.777306 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.777307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.777307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.777444 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.777839 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:29.777847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:29.777932 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 05:19:29.778046 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 05:19:29.778310 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":62,"thread_start_us":80,"threads_started":1}
I20260925 05:19:29.779373 3767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:29.779794 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.779649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40444)
0925 05:19:29.779704 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.779760 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 05:19:29.779954 3717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
I20260925 05:19:29.780408 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.780298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40458)
0925 05:19:29.780349 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.780380 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 05:19:29.780525 3767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:36621: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
I20260925 05:19:29.790738 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:36621
--webserver_interface=127.13.160.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:19:29.829819 3767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } attempt: 1
I20260925 05:19:29.830391 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.830240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40470)
0925 05:19:29.830305 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.830356 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 05:19:29.830817 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.830726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40484)
0925 05:19:29.830770 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.830799 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 05:19:29.830962 3767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:36621: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
W20260925 05:19:29.876320 3774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:29.876586 3774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:29.876605 3774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:29.876631 3774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:29.878021 3774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:29.878096 3774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:29.878113 3774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:29.878126 3774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:29.878139 3774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:29.879967 3774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:36621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.160.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:29.880306 3774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:29.880580 3774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:29.885921 3780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:29.885932 3779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:29.885923 3782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:29.886117 3774 server_base.cc:1034] running on GCE node
I20260925 05:19:29.886411 3774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:29.886780 3774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:29.887995 3774 hybrid_clock.cc:648] HybridClock initialized: now 1790313569887990 us; error 40 us; skew 500 ppm
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:29.890759 3774 init.cc:377] Logged in from keytab as kudu/127.13.160.189@KRBTEST.COM (short username kudu)
I20260925 05:19:29.891407 3774 webserver.cc:460] Webserver started at http://127.13.160.189:44545/ using document root <none> and password file <none>
I20260925 05:19:29.891664 3774 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:29.891739 3774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:29.891901 3774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:29.893074 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data/instance:
uuid: "237fdb345c524705ba6e78ad07761112"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "27f832be9b99df287142ec1982a034a2"
server_key_iv: "7af58305464aa2ee81886f3baa8b2a46"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.893474 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal/instance:
uuid: "237fdb345c524705ba6e78ad07761112"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "27f832be9b99df287142ec1982a034a2"
server_key_iv: "7af58305464aa2ee81886f3baa8b2a46"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.895015 3774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 05:19:29.895898 3789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:29.896075 3774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:29.896196 3774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "237fdb345c524705ba6e78ad07761112"
format_stamp: "Formatted at 2026-09-25 05:19:29 on dist-test-slave-9tgz"
server_key: "27f832be9b99df287142ec1982a034a2"
server_key_iv: "7af58305464aa2ee81886f3baa8b2a46"
server_key_version: "encryptionkey@0"
I20260925 05:19:29.896281 3774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:29.899156 3767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } attempt: 2
I20260925 05:19:29.899657 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.899510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40488)
0925 05:19:29.899578 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.899624 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 05:19:29.900190 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.900065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40500)
0925 05:19:29.900120 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.900150 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 05:19:29.900301 3767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:36621: Network error: Client connection negotiation failed: client connection to 127.13.160.189:36621: connect: Connection refused (error 111)
I20260925 05:19:29.906704 3774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:29.909000 3774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:29.909130 3774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:29.913471 3774 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.189:36621
I20260925 05:19:29.913507 3841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.189:36621 every 8 connection(s)
I20260925 05:19:29.913931 3774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
I20260925 05:19:29.915037 3842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:29.916047 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3774
I20260925 05:19:29.916126 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-1/wal/instance
I20260925 05:19:29.916325 13954 external_mini_cluster.cc:1428] Setting key 0dd21894b1b3f5025b68c633a88a1e88
I20260925 05:19:29.916986 3842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.188@KRBTEST.COM" created.
I20260925 05:19:29.927948 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.917922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.13.160.190:41957)
0925 05:19:29.917999 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:29.918002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:29.918304 (+ 302us) server_negotiation.cc:407] Connection header received
0925 05:19:29.919067 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:29.919069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:29.919087 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:29.919170 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:29.919837 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.920654 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.921724 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.922080 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.922178 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.924727 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:29.924729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:29.924729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:29.924737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:29.926270 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.926720 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.926721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.926722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.926783 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.927160 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.927161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.927162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.927329 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:29.927407 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:29.927792 (+ 385us) server_negotiation.cc:299] Negotiation successful
0925 05:19:29.927875 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 05:19:29.928260 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.917903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41957 (local address 127.0.0.1:59392)
0925 05:19:29.918132 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.918190 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:29.918939 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:29.919238 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:29.919242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:29.919514 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:29.919730 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.919734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.920765 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.920766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.921622 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.921624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.922193 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.922194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.922312 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.922984 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:29.923004 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:29.924608 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:29.926370 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.926387 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.926390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.926658 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.926847 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.926847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.926848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.927045 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.927418 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:29.927427 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:29.927594 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 05:19:29.927741 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 05:19:29.927951 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
I20260925 05:19:29.929006 3842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:29.937697 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.929347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40504)
0925 05:19:29.929403 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.929415 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:29.929460 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:29.929795 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:29.929795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:29.929927 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:29.929988 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.929990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.931049 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.931050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.931584 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.931585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.932335 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.932336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.932417 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.932926 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:29.932931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:29.934199 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:29.936581 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.936582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.936583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.936758 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.937013 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.937014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.937014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.937073 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.937431 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:29.937432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:29.937472 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 05:19:29.937524 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 05:19:29.937617 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 05:19:29.937830 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.929411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.13.160.189:36621)
0925 05:19:29.929609 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:29.929615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:29.929645 (+ 30us) server_negotiation.cc:407] Connection header received
0925 05:19:29.929673 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:29.929676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:29.929694 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:29.929785 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:29.930065 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.930949 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.931669 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.932229 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.932318 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.934294 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:29.934308 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:29.934308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:29.934318 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:29.936477 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.936854 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.936860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.936860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.936937 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.937146 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.937147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.937147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.937365 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:29.937439 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:29.937550 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 05:19:29.937673 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260925 05:19:29.938567 3842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:29.938988 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.938836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55416)
0925 05:19:29.938906 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.938965 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 05:19:29.939132 3793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
I20260925 05:19:29.939596 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.939494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55422)
0925 05:19:29.939546 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.939571 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 05:19:29.939760 3842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:34141: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
I20260925 05:19:29.946592 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:34141
--webserver_interface=127.13.160.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:45849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:19:29.989068 3842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } attempt: 1
I20260925 05:19:29.989625 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.989466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55426)
0925 05:19:29.989551 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.989594 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 05:19:29.989992 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.989894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55434)
0925 05:19:29.989943 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.989968 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 05:19:29.990083 3842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:34141: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
I20260925 05:19:29.991434 3767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } attempt: 3
Sep 25 05:19:29 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:29.999071 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.991843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.13.160.189:36621)
0925 05:19:29.991915 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:29.991917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:29.991926 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:29.991959 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:29.991960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:29.991972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:29.992056 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:29.992391 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.992907 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.993668 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.993907 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.993987 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.996292 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:29.996294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:29.996294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:29.996309 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:29.997802 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.998183 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.998185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.998185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.998248 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:29.998472 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:29.998473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:29.998473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:29.998603 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:29.998680 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:29.998850 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 05:19:29.998958 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 05:19:29.999086 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.991715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40506)
0925 05:19:29.991773 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.991785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:29.991835 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:29.992066 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:29.992067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:29.992236 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:29.992303 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.992304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.993001 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.993002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.993574 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:29.993576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:29.993985 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:29.993986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:29.994075 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:29.994611 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:29.994616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:29.996178 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:29.997902 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.997903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.997903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.998102 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.998329 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:29.998330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:29.998330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:29.998397 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:29.998682 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:29.998683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:29.998735 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 05:19:29.998825 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 05:19:29.998958 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 05:19:29.999536 3767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:29.999931 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:29.999803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55450)
0925 05:19:29.999858 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:29.999907 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 05:19:30.000303 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.000196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55452)
0925 05:19:30.000255 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.000286 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 05:19:30.000425 3767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:34141: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
I20260925 05:19:30.031569 3767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } attempt: 1
I20260925 05:19:30.032150 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.031994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55454)
0925 05:19:30.032073 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.032120 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 05:19:30.032562 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.032476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55458)
0925 05:19:30.032519 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.032546 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 05:19:30.032687 3767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:34141: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
W20260925 05:19:30.036347 3849 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:30.036602 3849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:30.036619 3849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:30.036643 3849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:30.037863 3849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:30.037907 3849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:30.037921 3849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:30.037935 3849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:30.037946 3849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:30.039727 3849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:45849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41957,127.13.160.189:36621,127.13.160.188:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.160.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:30.039964 3849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:30.040163 3849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:30.045385 3855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:30.045385 3857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:30.045429 3854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:30.045668 3849 server_base.cc:1034] running on GCE node
I20260925 05:19:30.045933 3849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:30.046296 3849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:30.047466 3849 hybrid_clock.cc:648] HybridClock initialized: now 1790313570047464 us; error 42 us; skew 500 ppm
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313570, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:30.050298 3849 init.cc:377] Logged in from keytab as kudu/127.13.160.188@KRBTEST.COM (short username kudu)
I20260925 05:19:30.050805 3849 webserver.cc:460] Webserver started at http://127.13.160.188:44027/ using document root <none> and password file <none>
I20260925 05:19:30.051077 3849 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:30.051155 3849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:30.051324 3849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:30.052523 3849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data/instance:
uuid: "09d1e165920245bb83490b04df8576f2"
format_stamp: "Formatted at 2026-09-25 05:19:30 on dist-test-slave-9tgz"
server_key: "c1b6999a927d0d5860db690d7a2066ae"
server_key_iv: "011c31e2188b2b2483c9ddd4f18de7a6"
server_key_version: "encryptionkey@0"
I20260925 05:19:30.052922 3849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal/instance:
uuid: "09d1e165920245bb83490b04df8576f2"
format_stamp: "Formatted at 2026-09-25 05:19:30 on dist-test-slave-9tgz"
server_key: "c1b6999a927d0d5860db690d7a2066ae"
server_key_iv: "011c31e2188b2b2483c9ddd4f18de7a6"
server_key_version: "encryptionkey@0"
I20260925 05:19:30.054391 3849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 05:19:30.055256 3864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:30.055444 3849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:30.055557 3849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "09d1e165920245bb83490b04df8576f2"
format_stamp: "Formatted at 2026-09-25 05:19:30 on dist-test-slave-9tgz"
server_key: "c1b6999a927d0d5860db690d7a2066ae"
server_key_iv: "011c31e2188b2b2483c9ddd4f18de7a6"
server_key_version: "encryptionkey@0"
I20260925 05:19:30.055641 3849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:30.058244 3842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } attempt: 2
I20260925 05:19:30.058751 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.058596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55464)
0925 05:19:30.058664 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.058708 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 05:19:30.059162 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.059059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55470)
0925 05:19:30.059111 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.059141 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 05:19:30.059271 3842 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:34141: Network error: Client connection negotiation failed: client connection to 127.13.160.188:34141: connect: Connection refused (error 111)
I20260925 05:19:30.082363 3849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:30.084586 3849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:30.084748 3849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:30.089040 3849 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.188:34141
I20260925 05:19:30.089077 3917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.188:34141 every 8 connection(s)
I20260925 05:19:30.089483 3849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
I20260925 05:19:30.090415 3918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:30.092098 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:30.092403 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 3849
I20260925 05:19:30.092497 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790313300911901-13954-0/minicluster-data/master-2/wal/instance
I20260925 05:19:30.092690 13954 external_mini_cluster.cc:1428] Setting key eb9cb3b0b85727724af14327500a4c84
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313570, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:30.103260 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.093101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.13.160.190:41957)
0925 05:19:30.093157 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.093159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.093442 (+ 283us) server_negotiation.cc:407] Connection header received
0925 05:19:30.093964 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.093965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.093980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.094038 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.094869 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.095640 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.096884 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.097148 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.097219 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.100000 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.100002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.100002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.100012 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.101660 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.102137 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.102139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.102140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.102211 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.102483 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.102484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.102484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.102622 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.102704 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.103043 (+ 339us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.103144 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:19:30.103500 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.093090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41957 (local address 127.0.0.1:59396)
0925 05:19:30.093346 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.093376 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:30.093870 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:30.094091 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:30.094093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:30.094492 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:30.094776 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.094784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.095734 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.095734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.096792 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.096795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.097236 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.097237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.097358 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.097983 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:30.097996 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:30.099889 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:30.101764 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.101774 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.101777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.102052 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.102283 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.102284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.102285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.102404 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.102695 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:30.102702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:30.102867 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 05:19:30.103021 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 05:19:30.103226 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":23,"thread_start_us":101,"threads_started":1}
I20260925 05:19:30.103739 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.093097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.13.160.188:34141)
0925 05:19:30.093342 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.093354 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.093392 (+ 38us) server_negotiation.cc:407] Connection header received
0925 05:19:30.093888 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.093901 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.093951 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.094052 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.094856 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.095999 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.097023 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.097435 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.097489 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.100478 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.100488 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.100488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.100496 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.102230 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.102640 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.102644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.102644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.102706 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.102956 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.102957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.102957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.103133 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.103307 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.103381 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.103468 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":201,"thread_start_us":99,"threads_started":1}
I20260925 05:19:30.104111 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:30.107870 3767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } attempt: 2
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313570, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:30.112960 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.104448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.13.160.189:36621)
0925 05:19:30.104491 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.104493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.104529 (+ 36us) server_negotiation.cc:407] Connection header received
0925 05:19:30.104609 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.104610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.104623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.104656 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.105019 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.105522 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.106244 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.106453 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.106506 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.108729 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.108730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.108731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.108739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.111006 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.111482 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.111483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.111484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.111558 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.111812 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.111814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.111814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.111986 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.112060 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.112778 (+ 718us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.112874 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 05:19:30.113379 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.104390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:36621 (local address 127.0.0.1:40510)
0925 05:19:30.104488 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.104500 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:30.104544 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:30.104724 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:30.104724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:30.104883 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:30.104957 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.104959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.105598 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.105599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.106145 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.106147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.106526 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.106527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.106607 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.107144 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:30.107148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:30.108615 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:30.111107 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.111109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.111110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.111409 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.111650 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.111651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.111651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.111725 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.112566 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:30.112567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:30.112625 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 05:19:30.113140 (+ 515us) client_negotiation.cc:240] Negotiation successful
0925 05:19:30.113257 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:30.113811 3918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:30.114506 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.104646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.13.160.190:41957)
0925 05:19:30.104708 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.104710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.104883 (+ 173us) server_negotiation.cc:407] Connection header received
0925 05:19:30.104912 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.104913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.104924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.104992 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.106101 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.106688 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.107553 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.107817 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.107910 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.110864 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.110867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.110867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.110877 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.112767 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.113200 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.113201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.113201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.113271 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.113507 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.113508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.113508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.113635 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.113846 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.114289 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.114379 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:19:30.117345 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.108267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.13.160.188:34141)
0925 05:19:30.108364 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.108366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.108540 (+ 174us) server_negotiation.cc:407] Connection header received
0925 05:19:30.108615 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.108617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.108627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.108681 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.108954 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.109425 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.111593 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.111901 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.112227 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.114140 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.114142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.114142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.114149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.116049 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.116394 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.116395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.116396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.116461 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.116693 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.116694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.116694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.116854 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.116926 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.117154 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.117247 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 05:19:30.117350 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.108218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55490)
0925 05:19:30.108477 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.108487 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:30.108556 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:30.108685 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:30.108686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:30.108837 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:30.108869 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.108871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.109507 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.109508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.111513 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.111515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.111996 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.111998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.112058 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.112446 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:30.112451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:30.114053 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:30.116115 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.116116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.116117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.116312 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.116546 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.116546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.116547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.116615 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.116945 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:30.116946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:30.117009 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 05:19:30.117123 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 05:19:30.117250 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243}
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313570, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:30.119885 3767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: Bootstrap starting.
I20260925 05:19:30.120549 3767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:30.120930 3767 log.cc:824] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:30.121918 3767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: No bootstrap required, opened a new log
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:30.122824 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.114151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55494)
0925 05:19:30.114198 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.114211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:30.114250 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:30.114598 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:30.114599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:30.114769 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:30.114819 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.114822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.115880 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.115881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.116490 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.116492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.117532 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.117534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.117607 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.118089 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:30.118094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:30.119635 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:30.121688 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.121689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.121690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.121926 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.122138 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.122138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.122139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.122193 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.122555 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:30.122556 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:30.122601 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 05:19:30.122646 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 05:19:30.122754 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 05:19:30.123276 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.114253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.13.160.188:34141)
0925 05:19:30.114463 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.114465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.114472 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:19:30.114490 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.114491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.114502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.114575 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.114950 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.115785 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.116591 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.117386 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.117764 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.119724 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.119725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.119726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.119734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.121595 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.122016 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.122017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.122018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.122070 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.122256 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.122257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.122257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.122472 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.122983 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.123028 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.123113 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":128,"threads_started":1}
I20260925 05:19:30.124241 3767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.124416 3767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2658b42dd79f4ba8b273ab74d92eb10c, State: Initialized, Role: FOLLOWER
I20260925 05:19:30.124513 3767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.124554 3918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: Bootstrap starting.
I20260925 05:19:30.124861 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.124927 3767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:30.124995 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.125301 3918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:30.125597 3918 log.cc:824] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:30.126540 3918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: No bootstrap required, opened a new log
I20260925 05:19:30.127029 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.115526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.13.160.189:36621)
0925 05:19:30.115567 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.115569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.115981 (+ 412us) server_negotiation.cc:407] Connection header received
0925 05:19:30.116028 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.116029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.116040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.116102 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.117058 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.117817 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.118833 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.119144 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.119231 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.122636 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.122638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.122638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.122647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.124804 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.125396 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.125397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.125398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.125467 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.125813 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.125814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.125815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.125974 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.126211 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.126260 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.126372 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 05:19:30.127126 3937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:30.127185 3937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:30.128502 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:30.128777 3918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.128957 3918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d1e165920245bb83490b04df8576f2, State: Initialized, Role: FOLLOWER
I20260925 05:19:30.129052 3918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.129395 3918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:30.129585 3741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2658b42dd79f4ba8b273ab74d92eb10c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59400
I20260925 05:19:30.129727 3741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 05:19:30.129769 3741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:30.128119 3698 master_runner.cc:186] Error getting master registration for 127.13.160.189:36621: OK, instance_id { permanent_uuid: "237fdb345c524705ba6e78ad07761112" instance_seqno: 1790313569912194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:19:30.130759 3698 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:19:30.130740 3741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.130991 3741 leader_election.cc:288] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [CANDIDATE]: Term 1 election: Requested vote from peers 237fdb345c524705ba6e78ad07761112 (127.13.160.189:36621), 09d1e165920245bb83490b04df8576f2 (127.13.160.188:34141)
I20260925 05:19:30.131270 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "09d1e165920245bb83490b04df8576f2"
I20260925 05:19:30.131443 3891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [term 0 FOLLOWER]: Advancing to term 1
W20260925 05:19:30.131898 3950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:30.131980 3950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:30.132885 3891 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2658b42dd79f4ba8b273ab74d92eb10c in term 1.
I20260925 05:19:30.133083 3717 leader_election.cc:302] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d1e165920245bb83490b04df8576f2, 2658b42dd79f4ba8b273ab74d92eb10c; no voters:
I20260925 05:19:30.133221 3925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:30.133387 3925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [term 1 LEADER]: Becoming Leader. State: Replica: 2658b42dd79f4ba8b273ab74d92eb10c, State: Running, Role: LEADER
I20260925 05:19:30.133498 3925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.133922 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2658b42dd79f4ba8b273ab74d92eb10c. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.134009 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:30.134521 3958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:30.134786 3958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:30.133586 3849 master_runner.cc:186] Error getting master registration for 127.13.160.189:36621: OK, instance_id { permanent_uuid: "237fdb345c524705ba6e78ad07761112" instance_seqno: 1790313569912194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:19:30.136049 3849 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:19:30.133601 3939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.136281 3939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.138139 3891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [term 1 FOLLOWER]: Refusing update from remote peer 2658b42dd79f4ba8b273ab74d92eb10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:30.138543 3925 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:30.140321 3939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2658b42dd79f4ba8b273ab74d92eb10c. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.140507 3939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.149214 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.149387 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:30.150108 3939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.150301 3939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.150398 3842 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } attempt: 3
I20260925 05:19:30.150867 3959 mvcc.cc:205] Tried to move back new op lower bound from 7333124383282642944 to 7333124383267504128. Current Snapshot: MvccSnapshot[applied={T|T < 7333124383282642944}]
Sep 25 05:19:30 dist-test-slave-9tgz krb5kdc[3685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313569, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:30.155997 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.139369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.13.160.188:34141)
0925 05:19:30.139440 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.139442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.140046 (+ 604us) server_negotiation.cc:407] Connection header received
0925 05:19:30.140447 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.140448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.140461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.140491 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.141155 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.141924 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.144463 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.144822 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.144902 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.147529 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.147531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.147531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.147541 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.150654 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.152871 (+ 2217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.152873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.152873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.152956 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.154189 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.154191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.154192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.154357 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.154451 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.155714 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.155887 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 05:19:30.161151 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.150735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.13.160.188:34141)
0925 05:19:30.150774 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.150776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.150813 (+ 37us) server_negotiation.cc:407] Connection header received
0925 05:19:30.150908 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.150909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.150920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.150975 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.151295 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.152018 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.153004 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.153336 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.153408 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.156832 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.156833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.156834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.156844 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.159851 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.160294 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.160296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.160296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.160361 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.160564 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.160565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.160566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.160706 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.160789 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.160959 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.161042 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:30.161162 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.150707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:34141 (local address 127.0.0.1:55516)
0925 05:19:30.150768 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:30.150780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:30.150831 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:30.150989 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:30.150990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:30.151169 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:30.151223 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.151226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.152098 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.152099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.152912 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:30.152915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.153418 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.153419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:30.153475 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.153989 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:30.154005 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:30.155662 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:30.159938 (+ 4276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.159940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.159941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.160209 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.160442 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:30.160442 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:30.160443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:30.160510 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:30.160783 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:30.160784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:30.160836 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 05:19:30.160922 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 05:19:30.161043 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 05:19:30.161959 3958 catalog_manager.cc:1347] Generated new cluster ID: 3e1b18344e0e4ed39afe00a99177549b
I20260925 05:19:30.162017 3958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:30.162947 3842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: Bootstrap starting.
I20260925 05:19:30.163746 3842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:30.164052 3842 log.cc:824] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:30.176059 3842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: No bootstrap required, opened a new log
I20260925 05:19:30.178372 3842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.178565 3842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237fdb345c524705ba6e78ad07761112, State: Initialized, Role: FOLLOWER
I20260925 05:19:30.178668 3842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } }
I20260925 05:19:30.179116 3842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:30.180577 3816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "237fdb345c524705ba6e78ad07761112"
I20260925 05:19:30.180735 3816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:30.181720 3816 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2658b42dd79f4ba8b273ab74d92eb10c in term 1.
I20260925 05:19:30.181911 3815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [term 1 FOLLOWER]: Refusing update from remote peer 2658b42dd79f4ba8b273ab74d92eb10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:30.182305 3956 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 05:19:30.183341 3979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:30.183400 3979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:30.185142 3968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.185264 3968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.185333 3968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.185391 3968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.186758 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.139469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.13.160.190:41957)
0925 05:19:30.139510 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.139512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.139887 (+ 375us) server_negotiation.cc:407] Connection header received
0925 05:19:30.140094 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.140096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.140107 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.140149 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.147595 (+ 7446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.148783 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.159289 (+ 10506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.159607 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.159652 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.179043 (+ 19391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.179046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.179046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.179056 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.180890 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.184699 (+ 3809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.184701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.184701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.184772 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.185848 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.185849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.185849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.186000 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.186098 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.186507 (+ 409us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.186627 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:30.186949 3984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2658b42dd79f4ba8b273ab74d92eb10c. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.187042 3984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:30.188113 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:30.139111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.13.160.189:36621)
0925 05:19:30.139157 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.139159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.139408 (+ 249us) server_negotiation.cc:407] Connection header received
0925 05:19:30.139599 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.139600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.139622 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.139654 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.141590 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.142366 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.156961 (+ 14595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:30.157324 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:30.157371 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:30.176629 (+ 19258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:30.176631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:30.176631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:30.176641 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:30.178531 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.184355 (+ 5824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.184357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.184357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.184436 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:30.185471 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:30.185472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:30.185473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:30.185643 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:30.185777 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:30.187908 (+ 2131us) server_negotiation.cc:299] Negotiation successful
0925 05:19:30.188014 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:19:30.196638 3958 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:30.195899 3925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2658b42dd79f4ba8b273ab74d92eb10c" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41957 } } peers { permanent_uuid: "237fdb345c524705ba6e78ad07761112" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 36621 } } peers { permanent_uuid: "09d1e165920245bb83490b04df8576f2" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 34141 } } }
I20260925 05:19:30.196836 3925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:30.197996 3958 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:30.208746 3958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2658b42dd79f4ba8b273ab74d92eb10c: Generated new TSK 0
I20260925 05:19:30.209090 3958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:19:30.243091 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.241256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.13.160.189:36621)
0925 05:19:30.241307 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.241310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.242324 (+ 1014us) server_negotiation.cc:407] Connection header received
0925 05:19:30.242439 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.242440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.242453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.242486 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.243011 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40550: BlockingRecv error: recv got EOF from 127.0.0.1:40550 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 05:19:30.244058 3921 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.241648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.13.160.188:34141)
0925 05:19:30.241692 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.241694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.243322 (+ 1628us) server_negotiation.cc:407] Connection header received
0925 05:19:30.243490 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.243491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.243504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.243535 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.243989 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55530: BlockingRecv error: recv got EOF from 127.0.0.1:55530 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:30.244735 3771 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.241986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.13.160.190:41957)
0925 05:19:30.242026 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.242028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.243624 (+ 1596us) server_negotiation.cc:407] Connection header received
0925 05:19:30.244322 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.244323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.244334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.244366 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.244673 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59424: BlockingRecv error: recv got EOF from 127.0.0.1:59424 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 05:19:30.247437 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.246538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.13.160.189:36621)
0925 05:19:30.246577 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.246579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.246816 (+ 237us) server_negotiation.cc:407] Connection header received
0925 05:19:30.246969 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.246970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.246992 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.247023 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.247381 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40556: BlockingRecv error: recv got EOF from 127.0.0.1:40556 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 05:19:30.250651 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.249748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.13.160.189:36621)
0925 05:19:30.249797 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.249799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.250082 (+ 283us) server_negotiation.cc:407] Connection header received
0925 05:19:30.250203 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.250204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.250216 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.250247 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.250597 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40566: BlockingRecv error: recv got EOF from 127.0.0.1:40566 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 05:19:30.254789 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.253888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.13.160.189:36621)
0925 05:19:30.253930 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.253932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.254207 (+ 275us) server_negotiation.cc:407] Connection header received
0925 05:19:30.254310 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.254311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.254322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.254352 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.254728 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40576: BlockingRecv error: recv got EOF from 127.0.0.1:40576 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 05:19:30.261539 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.260486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.13.160.189:36621)
0925 05:19:30.260535 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.260537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.260872 (+ 335us) server_negotiation.cc:407] Connection header received
0925 05:19:30.260980 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.260982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.260994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.261026 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.261473 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40584: BlockingRecv error: recv got EOF from 127.0.0.1:40584 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:30.271970 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.270959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.13.160.189:36621)
0925 05:19:30.271007 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.271010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.271328 (+ 318us) server_negotiation.cc:407] Connection header received
0925 05:19:30.271435 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.271436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.271449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.271482 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.271892 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40592: BlockingRecv error: recv got EOF from 127.0.0.1:40592 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 05:19:30.290362 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.289345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.13.160.189:36621)
0925 05:19:30.289394 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.289396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.289715 (+ 319us) server_negotiation.cc:407] Connection header received
0925 05:19:30.289823 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.289825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.289838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.289870 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.290298 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40604: BlockingRecv error: recv got EOF from 127.0.0.1:40604 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 05:19:30.324903 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.323843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40614 (local address 127.13.160.189:36621)
0925 05:19:30.323893 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.323895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.324250 (+ 355us) server_negotiation.cc:407] Connection header received
0925 05:19:30.324361 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.324362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.324375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.324406 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.324837 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40614: BlockingRecv error: recv got EOF from 127.0.0.1:40614 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 05:19:30.391109 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.390319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40628 (local address 127.13.160.189:36621)
0925 05:19:30.390452 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.390454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.390470 (+ 16us) server_negotiation.cc:407] Connection header received
0925 05:19:30.390551 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.390552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.390563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.390617 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.391036 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40628: BlockingRecv error: recv got EOF from 127.0.0.1:40628 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260925 05:19:30.521140 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.520353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.13.160.189:36621)
0925 05:19:30.520421 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.520422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.520602 (+ 180us) server_negotiation.cc:407] Connection header received
0925 05:19:30.520650 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.520651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.520661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.520728 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.521045 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40638: BlockingRecv error: recv got EOF from 127.0.0.1:40638 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 05:19:30.779292 3847 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:30.778523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.13.160.189:36621)
0925 05:19:30.778588 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:30.778591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:30.778601 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:19:30.778704 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:30.778705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:30.778723 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:30.778794 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:30.779221 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40646: BlockingRecv error: recv got EOF from 127.0.0.1:40646 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 05:19:31.132473 3950 catalog_manager.cc:1259] Loaded cluster ID: 3e1b18344e0e4ed39afe00a99177549b
I20260925 05:19:31.132586 3950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: loading cluster ID for follower catalog manager: success
I20260925 05:19:31.133424 3950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: acquiring CA information for follower catalog manager: success
I20260925 05:19:31.133774 3950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09d1e165920245bb83490b04df8576f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:19:31.183853 3979 catalog_manager.cc:1259] Loaded cluster ID: 3e1b18344e0e4ed39afe00a99177549b
I20260925 05:19:31.183943 3979 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: loading cluster ID for follower catalog manager: success
I20260925 05:19:31.184743 3979 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: acquiring CA information for follower catalog manager: success
I20260925 05:19:31.185004 3979 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 237fdb345c524705ba6e78ad07761112: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:19:31.295361 4052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:31.292675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.13.160.189:36621)
0925 05:19:31.292773 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:31.292774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:31.292785 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:31.292855 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:31.292856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:31.292944 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:31.293034 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:31.293195 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.293739 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.294514 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.294713 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.294768 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:31.294988 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:31.295135 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:31.295202 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:31.295270 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 05:19:31.295290 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":53,"threads_started":1}
I20260925 05:19:31.299974 4058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:31.297345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.13.160.188:34141)
0925 05:19:31.297447 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:31.297449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:31.297460 (+ 11us) server_negotiation.cc:407] Connection header received
0925 05:19:31.297532 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:31.297533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:31.297572 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:31.297645 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:31.297738 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.298232 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.299062 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.299266 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.299367 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:31.299573 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:31.299699 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:31.299789 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:31.299840 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 05:19:31.299865 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":51,"threads_started":1}
I20260925 05:19:31.305045 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:31.301871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59430 (local address 127.13.160.190:41957)
0925 05:19:31.301996 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:31.301999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:31.302009 (+ 10us) server_negotiation.cc:407] Connection header received
0925 05:19:31.302137 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:31.302138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:31.302221 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:31.302312 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:31.302483 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.303265 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304085 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.304368 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304434 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:31.304658 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:31.304793 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:31.304860 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:31.304906 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 05:19:31.304937 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260925 05:19:31.305159 4058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:31.302100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.13.160.188:34141)
0925 05:19:31.302137 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:31.302138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:31.302445 (+ 307us) server_negotiation.cc:407] Connection header received
0925 05:19:31.302488 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:31.302489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:31.302498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:31.302552 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:31.302673 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.303264 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304319 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.304584 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304666 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:31.304791 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:31.304824 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:31.304872 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:31.305017 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 05:19:31.305045 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:19:31.305178 4052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:31.301810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.13.160.189:36621)
0925 05:19:31.301880 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:31.301882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:31.302049 (+ 167us) server_negotiation.cc:407] Connection header received
0925 05:19:31.302116 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:31.302117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:31.302132 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:31.302210 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:31.302345 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.303143 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304271 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:31.304588 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:31.304667 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:31.304848 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:31.304881 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:31.304926 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:31.305070 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 05:19:31.305095 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:19:31.307269 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3698
I20260925 05:19:31.313448 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3774
I20260925 05:19:31.318740 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 3849
2026-09-25T05:19:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2853 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 05:19:31 dist-test-slave-9tgz krb5kdc[4069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 05:19:31 dist-test-slave-9tgz krb5kdc[4069](info): set up 2 sockets
Sep 25 05:19:31 dist-test-slave-9tgz krb5kdc[4069](info): commencing operation
krb5kdc: starting...
W20260925 05:19:32.368257 13954 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/test-user.keytab.
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T05:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T05:19:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.160.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
Entry for principal kudu/127.13.160.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab.
I20260925 05:19:32.495078 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41165
--webserver_interface=127.13.160.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 05:19:32.578868 4082 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:32.579221 4082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:32.579254 4082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:32.579293 4082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:32.580607 4082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:32.580688 4082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:32.580720 4082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:32.580751 4082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:32.580780 4082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:32.582408 4082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.190.keytab
--principal=kudu/127.13.160.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.190:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.160.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:32.582731 4082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:32.583003 4082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 05:19:32.588356 4082 server_base.cc:1034] running on GCE node
W20260925 05:19:32.588336 4088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:32.588327 4087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:32.588335 4090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:32.588856 4082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:32.589193 4082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:32.590374 4082 hybrid_clock.cc:648] HybridClock initialized: now 1790313572590351 us; error 37 us; skew 500 ppm
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:32.592983 4082 init.cc:377] Logged in from keytab as kudu/127.13.160.190@KRBTEST.COM (short username kudu)
I20260925 05:19:32.593501 4082 webserver.cc:460] Webserver started at http://127.13.160.190:40463/ using document root <none> and password file <none>
I20260925 05:19:32.593750 4082 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:32.593822 4082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:32.593978 4082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:32.595213 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data/instance:
uuid: "c148d443950d40de90dd27c6827b6ae0"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "f21eeec38e81875a5f171d2077769866"
server_key_iv: "3520f4793e9b305da0f97f0936009d69"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.595592 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance:
uuid: "c148d443950d40de90dd27c6827b6ae0"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "f21eeec38e81875a5f171d2077769866"
server_key_iv: "3520f4793e9b305da0f97f0936009d69"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.597048 4082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 05:19:32.597966 4097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:32.598183 4082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:32.598299 4082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal
uuid: "c148d443950d40de90dd27c6827b6ae0"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "f21eeec38e81875a5f171d2077769866"
server_key_iv: "3520f4793e9b305da0f97f0936009d69"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.598380 4082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:32.607244 4082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:32.609319 4082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:32.609505 4082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:32.613811 4082 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.190:41165
I20260925 05:19:32.613844 4149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.190:41165 every 8 connection(s)
I20260925 05:19:32.614234 4082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/data/info.pb
I20260925 05:19:32.615231 4150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:32.616837 4150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.620304 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 4082
I20260925 05:19:32.620425 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-0/wal/instance
I20260925 05:19:32.620620 13954 external_mini_cluster.cc:1428] Setting key d834c4e9a4abad70753d370a5d5cb24c
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:32.629206 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.617527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41165 (local address 127.0.0.1:33066)
0925 05:19:32.617701 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.617771 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.618238 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.618518 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.618521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.618788 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.619003 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.619007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.620025 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.620026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.621221 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.621225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.621738 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.621739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.621827 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.623558 (+ 1731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.623567 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.625311 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.627622 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.627629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.627631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.627850 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.628132 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.628133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.628133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.628215 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.628725 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.628730 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.628790 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.628864 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.629004 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260925 05:19:32.629209 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.617588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.13.160.190:41165)
0925 05:19:32.617754 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.617762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.617837 (+ 75us) server_negotiation.cc:407] Connection header received
0925 05:19:32.618379 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.618381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.618399 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.618490 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.619094 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.619924 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.621308 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.621635 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.621722 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.625422 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.625432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.625433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.625442 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.627520 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.627943 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.627947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.627948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.628060 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.628287 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.628288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.628288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.628643 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.628744 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.628934 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.629092 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.13.160.189@KRBTEST.COM; defaulting to no policy
I20260925 05:19:32.630206 4150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.630597 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.630474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40046)
0925 05:19:32.630525 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.630573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 05:19:32.630748 4101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
I20260925 05:19:32.631208 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.631105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40056)
0925 05:19:32.631157 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.631189 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 05:19:32.631351 4150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:33749: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
Entry for principal kudu/127.13.160.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab.
I20260925 05:19:32.650272 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:33749
--webserver_interface=127.13.160.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:19:32.680629 4150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } attempt: 1
I20260925 05:19:32.681161 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.681018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40060)
0925 05:19:32.681086 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.681130 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 05:19:32.681571 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.681478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40062)
0925 05:19:32.681529 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.681555 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 05:19:32.681667 4150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:33749: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
W20260925 05:19:32.733655 4157 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:32.733908 4157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:32.733927 4157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:32.733950 4157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:32.735313 4157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:32.735368 4157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:32.735383 4157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:32.735397 4157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:32.735409 4157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:32.737048 4157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.189.keytab
--principal=kudu/127.13.160.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.189:33749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.160.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:32.737349 4157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:32.737617 4157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:32.742769 4162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:32.742983 4157 server_base.cc:1034] running on GCE node
W20260925 05:19:32.742769 4163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:32.742787 4165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:32.743364 4157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:32.743734 4157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:32.744925 4157 hybrid_clock.cc:648] HybridClock initialized: now 1790313572744901 us; error 38 us; skew 500 ppm
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:32.747651 4157 init.cc:377] Logged in from keytab as kudu/127.13.160.189@KRBTEST.COM (short username kudu)
I20260925 05:19:32.748083 4157 webserver.cc:460] Webserver started at http://127.13.160.189:46863/ using document root <none> and password file <none>
I20260925 05:19:32.748281 4157 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:32.748322 4157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:32.748414 4157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:32.749502 4157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data/instance:
uuid: "f64d41d8880d471195663e39e29f2510"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "acb3d96d985710bcc2f65042a2bf0ee2"
server_key_iv: "4f8d96e1d1f52cc757e30d614ab5a67f"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.749825 4157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal/instance:
uuid: "f64d41d8880d471195663e39e29f2510"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "acb3d96d985710bcc2f65042a2bf0ee2"
server_key_iv: "4f8d96e1d1f52cc757e30d614ab5a67f"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.749850 4150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } attempt: 2
I20260925 05:19:32.750283 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.750144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40076)
0925 05:19:32.750203 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.750236 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 05:19:32.750698 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.750598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40090)
0925 05:19:32.750627 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.750651 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260925 05:19:32.750803 4150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.189:33749: Network error: Client connection negotiation failed: client connection to 127.13.160.189:33749: connect: Connection refused (error 111)
I20260925 05:19:32.751611 4157 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 05:19:32.752389 4172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:32.752565 4157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 05:19:32.752648 4157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal
uuid: "f64d41d8880d471195663e39e29f2510"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "acb3d96d985710bcc2f65042a2bf0ee2"
server_key_iv: "4f8d96e1d1f52cc757e30d614ab5a67f"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.752748 4157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:32.775575 4157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:32.777583 4157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:32.777750 4157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:32.782023 4157 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.189:33749
I20260925 05:19:32.782091 4225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.189:33749 every 8 connection(s)
I20260925 05:19:32.782486 4157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/data/info.pb
I20260925 05:19:32.783447 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:32.785115 4226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.785640 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 4157
I20260925 05:19:32.785750 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-1/wal/instance
I20260925 05:19:32.786067 13954 external_mini_cluster.cc:1428] Setting key 8699f347b27d3a96e8dc7a68889524c8
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.160.188@KRBTEST.COM; defaulting to no policy
I20260925 05:19:32.796022 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.786033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.13.160.190:41165)
0925 05:19:32.786127 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.786130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.786414 (+ 284us) server_negotiation.cc:407] Connection header received
0925 05:19:32.786915 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.786917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.786936 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.787014 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.787658 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.788438 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.789988 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.790268 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.790327 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.793022 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.793024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.793024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.793032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.794519 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.794987 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.794989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.794989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.795055 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.795349 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.795349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.795350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.795515 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.795585 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.795874 (+ 289us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.795952 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 05:19:32.796216 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.786021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41165 (local address 127.0.0.1:33074)
0925 05:19:32.786289 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.786321 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.786799 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.787076 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.787080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.787347 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.787552 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.787556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.788546 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.788547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.789890 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.789894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.790371 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.790372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.790539 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.791219 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.791232 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.792910 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.794620 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.794629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.794632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.794879 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.795146 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.795147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.795147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.795272 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.795587 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.795595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.795741 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.795846 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.795988 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":90,"threads_started":1}
I20260925 05:19:32.796839 4226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.160.188@KRBTEST.COM" created.
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:32.805361 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.797203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40096)
0925 05:19:32.797237 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.797248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.797294 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.797663 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.797664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.797787 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.797845 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.797846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.799178 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.799179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.799808 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.799810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.800331 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.800332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.800398 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.800876 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.800881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.802238 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.804211 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.804212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.804212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.804389 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.804616 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.804616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.804617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.804678 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.804996 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.804997 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.805037 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.805099 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.805212 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 05:19:32.805397 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.797282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.13.160.189:33749)
0925 05:19:32.797486 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.797491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.797520 (+ 29us) server_negotiation.cc:407] Connection header received
0925 05:19:32.797545 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.797548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.797564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.797644 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.797921 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.799069 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.799903 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.800249 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.800335 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.802345 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.802356 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.802357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.802364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.804104 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.804485 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.804489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.804490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.804549 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.804741 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.804741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.804742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.804923 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.805023 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.805138 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.805275 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260925 05:19:32.806232 4226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.806640 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.806516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48686)
0925 05:19:32.806569 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.806617 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 05:19:32.806808 4176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
I20260925 05:19:32.807322 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.807223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48694)
0925 05:19:32.807277 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.807303 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 05:19:32.807432 4226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:36405: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
Entry for principal kudu/127.13.160.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab.
I20260925 05:19:32.812116 13954 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskq3iYZX/build/release/bin/kudu
/tmp/dist-test-taskq3iYZX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:36405
--webserver_interface=127.13.160.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.160.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 05:19:32.842049 4150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } attempt: 3
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:32.849807 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.842466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40112)
0925 05:19:32.842535 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.842550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.842634 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.842803 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.842803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.843037 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.843106 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.843109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.843796 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.843797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.844335 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.844337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.844740 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.844741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.844841 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.845416 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.845422 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.846998 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.848666 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.848667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.848668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.848857 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.849083 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.849084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.849084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.849164 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.849440 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.849442 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.849493 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.849582 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.849697 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 05:19:32.849807 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.842543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.13.160.189:33749)
0925 05:19:32.842620 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.842623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.842635 (+ 12us) server_negotiation.cc:407] Connection header received
0925 05:19:32.842704 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.842705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.842715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.842798 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.843199 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.843693 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.844430 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.844661 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.844748 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.847092 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.847093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.847094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.847108 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.848511 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.848950 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.848952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.848952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.849014 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.849232 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.849233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.849234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.849364 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.849438 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.849612 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.849700 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 05:19:32.850500 4150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.850931 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.850782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48704)
0925 05:19:32.850830 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.850864 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 05:19:32.851215 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.851157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48714)
0925 05:19:32.851181 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.851203 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 05:19:32.851318 4150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:36405: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
I20260925 05:19:32.856633 4226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } attempt: 1
I20260925 05:19:32.857136 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.857004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48716)
0925 05:19:32.857065 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.857110 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 05:19:32.857460 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.857383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48726)
0925 05:19:32.857423 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.857445 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 05:19:32.857561 4226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:36405: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
I20260925 05:19:32.882484 4150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } attempt: 1
I20260925 05:19:32.883034 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.882862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48732)
0925 05:19:32.882946 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.882993 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 05:19:32.883435 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.883326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48748)
0925 05:19:32.883377 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.883411 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 05:19:32.883540 4150 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:36405: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
W20260925 05:19:32.898272 4233 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 05:19:32.898525 4233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 05:19:32.898545 4233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 05:19:32.898604 4233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 05:19:32.900297 4233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 05:19:32.900372 4233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 05:19:32.900389 4233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 05:19:32.900403 4233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 05:19:32.900415 4233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 05:19:32.902316 4233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.160.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.160.190:41165,127.13.160.189:33749,127.13.160.188:36405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/krb5kdc/kudu_127.13.160.188.keytab
--principal=kudu/127.13.160.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.160.188:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.160.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 04:15:06 UTC on ccycloud.quasar-mlmkqt.root.comops.site
build id 4671826
I20260925 05:19:32.902623 4233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 05:19:32.902959 4233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 05:19:32.907984 4241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:32.908021 4239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 05:19:32.908011 4238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 05:19:32.908320 4233 server_base.cc:1034] running on GCE node
I20260925 05:19:32.908596 4233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 05:19:32.908957 4233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 05:19:32.910132 4233 hybrid_clock.cc:648] HybridClock initialized: now 1790313572910115 us; error 30 us; skew 500 ppm
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 05:19:32.912921 4233 init.cc:377] Logged in from keytab as kudu/127.13.160.188@KRBTEST.COM (short username kudu)
I20260925 05:19:32.913512 4233 webserver.cc:460] Webserver started at http://127.13.160.188:34105/ using document root <none> and password file <none>
I20260925 05:19:32.913802 4233 fs_manager.cc:359] Metadata directory not provided
I20260925 05:19:32.913883 4233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 05:19:32.914055 4233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 05:19:32.915534 4233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data/instance:
uuid: "22c9fb4ade074aa59c38d876d0ca1c19"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "4898134437051fe788b3d051dcc50642"
server_key_iv: "d6d8b4295efd5e8b8ea1b85e13a2c6b4"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.915942 4233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal/instance:
uuid: "22c9fb4ade074aa59c38d876d0ca1c19"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "4898134437051fe788b3d051dcc50642"
server_key_iv: "d6d8b4295efd5e8b8ea1b85e13a2c6b4"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.917469 4233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 05:19:32.918351 4248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 05:19:32.918565 4233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 05:19:32.918689 4233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data,/tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal
uuid: "22c9fb4ade074aa59c38d876d0ca1c19"
format_stamp: "Formatted at 2026-09-25 05:19:32 on dist-test-slave-9tgz"
server_key: "4898134437051fe788b3d051dcc50642"
server_key_iv: "d6d8b4295efd5e8b8ea1b85e13a2c6b4"
server_key_version: "encryptionkey@0"
I20260925 05:19:32.918779 4233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 05:19:32.925712 4226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } attempt: 2
I20260925 05:19:32.926270 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.926125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48754)
0925 05:19:32.926187 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.926236 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 05:19:32.926654 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.926571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48760)
0925 05:19:32.926615 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.926639 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 05:19:32.926781 4226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.160.188:36405: Network error: Client connection negotiation failed: client connection to 127.13.160.188:36405: connect: Connection refused (error 111)
I20260925 05:19:32.936494 4233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 05:19:32.939204 4233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 05:19:32.939371 4233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 05:19:32.943804 4233 rpc_server.cc:307] RPC server started. Bound to: 127.13.160.188:36405
I20260925 05:19:32.943842 4301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.160.188:36405 every 8 connection(s)
I20260925 05:19:32.944276 4233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/data/info.pb
I20260925 05:19:32.945427 4302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 05:19:32.947443 13954 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskq3iYZX/build/release/bin/kudu as pid 4233
I20260925 05:19:32.947559 13954 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskq3iYZX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790313300911901-13954-0/minicluster-data/master-2/wal/instance
I20260925 05:19:32.947767 13954 external_mini_cluster.cc:1428] Setting key 62b2396e1d2f35cda299fa7bf6ef2c68
I20260925 05:19:32.947443 4302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:32.958684 4150 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } attempt: 2
I20260925 05:19:32.959048 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.948502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.13.160.190:41165)
0925 05:19:32.948579 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.948582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.948748 (+ 166us) server_negotiation.cc:407] Connection header received
0925 05:19:32.949265 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.949267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.949281 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.949347 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.950207 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.951021 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.952255 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.952504 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.952576 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.955502 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.955505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.955505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.955515 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.957256 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.957738 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.957739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.957739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.957795 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.958132 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.958132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.958133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.958304 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.958383 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.958755 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.958958 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 05:19:32.959228 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.948338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.190:41165 (local address 127.0.0.1:33076)
0925 05:19:32.948625 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.948661 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.949160 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.949396 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.949399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.949815 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.950080 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.950097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.951126 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.951127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.952162 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.952165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.952594 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.952595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.952779 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.953381 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.953402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.955325 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.957349 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.957354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.957357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.957647 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.957863 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.957864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.957864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.958023 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.958378 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.958383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.958592 (+ 209us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.958726 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.958949 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":56,"thread_start_us":108,"threads_started":1}
I20260925 05:19:32.959580 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.948338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.13.160.188:36405)
0925 05:19:32.948824 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.948828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.948858 (+ 30us) server_negotiation.cc:407] Connection header received
0925 05:19:32.949179 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.949190 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.949241 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.949357 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.950217 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.951396 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.952411 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.952868 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.952922 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.955938 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.955948 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.955948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.955955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.958018 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.958445 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.958449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.958449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.958510 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.958757 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.958758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.958759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.958986 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.959166 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.959390 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.959469 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":447,"thread_start_us":116,"threads_started":1}
I20260925 05:19:32.960088 4302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.190@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.190@KRBTEST.COM
I20260925 05:19:32.968302 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.958990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48786)
0925 05:19:32.959116 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.959127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.959165 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.959684 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.959686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.959848 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.959898 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.959900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.960785 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.960786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.961588 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.961591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.962061 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.962062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.962127 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.962664 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.962669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.964368 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.967015 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.967016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.967309 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.967541 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.967542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.967611 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.967946 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.967947 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.968000 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.968052 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.968158 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 05:19:32.968659 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.960464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.13.160.189:33749)
0925 05:19:32.960515 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.960516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.960524 (+ 8us) server_negotiation.cc:407] Connection header received
0925 05:19:32.960550 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.960551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.960562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.960624 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.961010 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.961707 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.962598 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.962952 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.963021 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.965625 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.965627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.965628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.965638 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.967372 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967826 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.967827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.967828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.967889 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.968093 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.968094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.968095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.968240 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.968304 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.968471 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.968563 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 05:19:32.968708 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.959240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.13.160.188:36405)
0925 05:19:32.959528 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.959530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.959537 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:19:32.959558 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.959558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.959570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.959673 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.959978 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.960715 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.961680 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.961980 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.962053 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.964478 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.964480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.964480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.964487 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.966934 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967407 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.967414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.967415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.967479 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967697 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.967698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.967699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.967859 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.968438 (+ 579us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.968479 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.968567 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":94,"threads_started":1}
I20260925 05:19:32.969015 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.960341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.189:33749 (local address 127.0.0.1:40126)
0925 05:19:32.960429 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.960438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.960470 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.960696 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.960697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.960855 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.960913 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.960915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.961787 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.961788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.962522 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.962525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.963031 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.963033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.963113 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.963716 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.963722 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.965528 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.967485 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.967487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.967730 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.967957 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.967958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.967959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.968018 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.968294 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.968295 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.968341 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.968779 (+ 438us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.968926 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260925 05:19:32.969169 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.960419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.13.160.190:41165)
0925 05:19:32.960457 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.960458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.960540 (+ 82us) server_negotiation.cc:407] Connection header received
0925 05:19:32.960607 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.960608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.960617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.960658 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.961240 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.961705 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.962553 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.962840 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.962936 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.965996 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.965998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.965998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.966006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.967757 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.968142 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.968143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.968143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.968194 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.968428 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.968428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.968429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.968564 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.968756 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.968994 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.969079 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 05:19:32.969388 4302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
I20260925 05:19:32.971058 4150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: Bootstrap starting.
I20260925 05:19:32.971871 4150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:32.972222 4150 log.cc:824] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:32.973150 4150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: No bootstrap required, opened a new log
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.188@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
Sep 25 05:19:32 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.160.189@KRBTEST.COM
I20260925 05:19:32.975652 4150 raft_consensus.cc:348] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.975849 4150 raft_consensus.cc:729] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c148d443950d40de90dd27c6827b6ae0, State: Initialized, Role: FOLLOWER
I20260925 05:19:32.975961 4150 consensus_queue.cc:260] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.976389 4150 sys_catalog.cc:564] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:32.976320 4309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:32.976730 4309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:32.977954 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.969655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48796)
0925 05:19:32.969734 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:32.969744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:32.969783 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:32.970095 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:32.970096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:32.970226 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:32.970262 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.970264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.971196 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.971197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.971929 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:32.971932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.972473 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.972473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:32.972522 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.973063 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:32.973069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:32.974602 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:32.976624 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.976625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.976625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.976836 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.977091 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:32.977092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:32.977092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:32.977170 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:32.977578 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:32.977579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:32.977629 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 05:19:32.977701 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 05:19:32.977824 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 05:19:32.978391 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.969700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.13.160.188:36405)
0925 05:19:32.969839 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.969841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.969847 (+ 6us) server_negotiation.cc:407] Connection header received
0925 05:19:32.969864 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.969864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.969873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.969917 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.970346 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.971105 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.972026 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.972350 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.972420 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.974704 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.974705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.974706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.974712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.976517 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.976932 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.976934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.976935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.977006 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.977252 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.977253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.977253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.977475 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.977550 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.978171 (+ 621us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.978260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260925 05:19:32.979262 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.970115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.13.160.189:33749)
0925 05:19:32.970164 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.970166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.970403 (+ 237us) server_negotiation.cc:407] Connection header received
0925 05:19:32.970470 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.970472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.970482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.970537 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.971116 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.971711 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.972481 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.972776 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.972839 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.975426 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.975429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.975429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.975437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:32.977463 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.977898 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.977900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.977900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.977966 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:32.978227 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:32.978228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:32.978229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:32.978384 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:32.978603 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:32.979071 (+ 468us) server_negotiation.cc:299] Negotiation successful
0925 05:19:32.979153 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 05:19:32.980175 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: Bootstrap starting.
I20260925 05:19:32.980959 4302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:32.981271 4302 log.cc:824] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:32.982193 4302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: No bootstrap required, opened a new log
W20260925 05:19:32.982625 4320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:32.982728 4320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:32.982872 13954 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 05:19:32.979818 4082 master_runner.cc:186] Error getting master registration for 127.13.160.189:33749: OK, instance_id { permanent_uuid: "f64d41d8880d471195663e39e29f2510" instance_seqno: 1790313572780754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:19:32.983057 4082 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:19:32.983971 4124 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c148d443950d40de90dd27c6827b6ae0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33082
I20260925 05:19:32.984104 4124 raft_consensus.cc:482] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 05:19:32.984161 4124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:32.984514 4302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.984725 4302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c9fb4ade074aa59c38d876d0ca1c19, State: Initialized, Role: FOLLOWER
I20260925 05:19:32.984826 4302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.985137 4124 raft_consensus.cc:504] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.985138 4323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:32.985211 4302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:32.985427 4124 leader_election.cc:288] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [CANDIDATE]: Term 1 election: Requested vote from peers f64d41d8880d471195663e39e29f2510 (127.13.160.189:33749), 22c9fb4ade074aa59c38d876d0ca1c19 (127.13.160.188:36405)
I20260925 05:19:32.986243 4275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c148d443950d40de90dd27c6827b6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "22c9fb4ade074aa59c38d876d0ca1c19"
I20260925 05:19:32.986354 4275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:32.987269 4275 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c148d443950d40de90dd27c6827b6ae0 in term 1.
I20260925 05:19:32.987519 4101 leader_election.cc:302] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22c9fb4ade074aa59c38d876d0ca1c19, c148d443950d40de90dd27c6827b6ae0; no voters:
I20260925 05:19:32.987676 4309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 05:19:32.987849 4309 raft_consensus.cc:686] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [term 1 LEADER]: Becoming Leader. State: Replica: c148d443950d40de90dd27c6827b6ae0, State: Running, Role: LEADER
I20260925 05:19:32.987957 4309 consensus_queue.cc:237] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:32.985297 4323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:19:32.989295 4339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 05:19:32.988628 4333 sys_catalog.cc:455] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c148d443950d40de90dd27c6827b6ae0. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
W20260925 05:19:32.989455 4339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:32.989430 4333 sys_catalog.cc:458] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:32.990439 4233 master_runner.cc:186] Error getting master registration for 127.13.160.189:33749: OK, instance_id { permanent_uuid: "f64d41d8880d471195663e39e29f2510" instance_seqno: 1790313572780754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 05:19:32.992956 4233 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 05:19:33.003238 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.988763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.13.160.189:33749)
0925 05:19:32.988827 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.988830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.992154 (+ 3324us) server_negotiation.cc:407] Connection header received
0925 05:19:32.992262 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.992263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.992276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.992308 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.993153 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.993971 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.996766 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.997092 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.997137 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.998926 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.998928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.998929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.998939 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:33.001354 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.002091 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.002093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.002093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.002165 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.002556 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.002557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.002558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.002704 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:33.002780 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:33.003047 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 05:19:33.003135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 05:19:33.003803 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.989014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.13.160.188:36405)
0925 05:19:32.989053 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.989056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.993158 (+ 4102us) server_negotiation.cc:407] Connection header received
0925 05:19:32.993218 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.993219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.993230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.993303 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.994001 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.994735 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.996720 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.997030 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:32.997107 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:32.999869 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:32.999871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:32.999871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:32.999880 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:33.001594 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.002304 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.002306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.002306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.002364 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.002531 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.002531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.002532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.002707 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:33.002810 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:33.003607 (+ 797us) server_negotiation.cc:299] Negotiation successful
0925 05:19:33.003702 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 05:19:33.004523 4344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 05:19:33.004801 4344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 05:19:33.007149 4275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [term 1 FOLLOWER]: Refusing update from remote peer c148d443950d40de90dd27c6827b6ae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:33.007527 4309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:33.009651 4323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c148d443950d40de90dd27c6827b6ae0. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.009766 4323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:33.012599 4309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.012717 4309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:33.012907 4346 mvcc.cc:205] Tried to move back new op lower bound from 7333124395033313280 to 7333124394959499264. Current Snapshot: MvccSnapshot[applied={T|T < 7333124395033313280}]
I20260925 05:19:33.013005 4323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.013108 4323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:33.013672 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:32.991806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.13.160.190:41165)
0925 05:19:32.991846 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:32.991848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:32.993302 (+ 1454us) server_negotiation.cc:407] Connection header received
0925 05:19:32.993392 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:32.993393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:32.993405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:32.993435 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:32.995024 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:32.995757 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.004375 (+ 8618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:33.004690 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.004768 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:33.007696 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:33.007698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:33.007698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:33.007707 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:33.011253 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.011774 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.011776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.011777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.011853 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.012192 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.012193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.012194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.012355 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:33.012452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:33.013448 (+ 996us) server_negotiation.cc:299] Negotiation successful
0925 05:19:33.013559 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 05:19:33.013825 4347 mvcc.cc:205] Tried to move back new op lower bound from 7333124395033313280 to 7333124394959499264. Current Snapshot: MvccSnapshot[applied={T|T < 7333124395033313280}]
I20260925 05:19:33.015404 4344 catalog_manager.cc:1347] Generated new cluster ID: de1c769ad7004538965ebe2fca8980bd
I20260925 05:19:33.015471 4344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 05:19:33.017928 4226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } attempt: 3
Sep 25 05:19:33 dist-test-slave-9tgz krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790313572, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.160.189@KRBTEST.COM for kudu/127.13.160.188@KRBTEST.COM
I20260925 05:19:33.028153 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:33.018310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.160.188:36405 (local address 127.0.0.1:48814)
0925 05:19:33.018371 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 05:19:33.018384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 05:19:33.018426 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 05:19:33.019267 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 05:19:33.019268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 05:19:33.019485 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 05:19:33.019552 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:33.019554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:33.020507 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.020509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:33.021389 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 05:19:33.021393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 05:19:33.021891 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.021893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 05:19:33.021959 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:33.023201 (+ 1242us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 05:19:33.023208 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 05:19:33.024867 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 05:19:33.027065 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:33.027066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:33.027067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:33.027290 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:33.027493 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 05:19:33.027493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 05:19:33.027494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 05:19:33.027546 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 05:19:33.027813 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 05:19:33.027814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 05:19:33.027860 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 05:19:33.027917 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 05:19:33.028048 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 05:19:33.028721 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:33.018372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.13.160.188:36405)
0925 05:19:33.018545 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.018547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.018556 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:33.018585 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.018586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.018597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.018649 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.019641 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:33.020400 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.021497 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:33.021804 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:33.021867 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:33.024968 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 05:19:33.024970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 05:19:33.024970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 05:19:33.024980 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 05:19:33.026948 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.027365 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.027366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.027367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.027428 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 05:19:33.027611 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 05:19:33.027612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 05:19:33.027612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 05:19:33.027741 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 05:19:33.027791 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:33.028537 (+ 746us) server_negotiation.cc:299] Negotiation successful
0925 05:19:33.028633 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260925 05:19:33.030628 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: Bootstrap starting.
I20260925 05:19:33.031399 4226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: Neither blocks nor log segments found. Creating new log.
I20260925 05:19:33.031692 4226 log.cc:824] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: Log is configured to *not* fsync() on all Append() calls
I20260925 05:19:33.034264 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: No bootstrap required, opened a new log
I20260925 05:19:33.036681 4226 raft_consensus.cc:348] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:33.036847 4226 raft_consensus.cc:729] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64d41d8880d471195663e39e29f2510, State: Initialized, Role: FOLLOWER
I20260925 05:19:33.036942 4226 consensus_queue.cc:260] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } }
I20260925 05:19:33.037355 4226 sys_catalog.cc:564] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 05:19:33.037408 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c148d443950d40de90dd27c6827b6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f64d41d8880d471195663e39e29f2510"
I20260925 05:19:33.037503 4200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [term 0 FOLLOWER]: Advancing to term 1
I20260925 05:19:33.038569 4200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c148d443950d40de90dd27c6827b6ae0 in term 1.
I20260925 05:19:33.038818 4199 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [term 1 FOLLOWER]: Refusing update from remote peer c148d443950d40de90dd27c6827b6ae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 05:19:33.037329 4352 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.039145 4309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 05:19:33.039290 4352 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:33.039520 4344 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 05:19:33.040289 4344 catalog_manager.cc:1504] Loading token signing keys...
I20260925 05:19:33.040757 4352 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c148d443950d40de90dd27c6827b6ae0. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.040915 4352 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 05:19:33.040980 4352 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.041060 4352 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 05:19:33.047565 4363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 05:19:33.047663 4363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 05:19:33.048297 4364 mvcc.cc:205] Tried to move back new op lower bound from 7333124395145641984 to 7333124394959499264. Current Snapshot: MvccSnapshot[applied={T|T < 7333124395033313280}]
I20260925 05:19:33.049163 4333 sys_catalog.cc:455] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c148d443950d40de90dd27c6827b6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c148d443950d40de90dd27c6827b6ae0" member_type: VOTER last_known_addr { host: "127.13.160.190" port: 41165 } } peers { permanent_uuid: "f64d41d8880d471195663e39e29f2510" member_type: VOTER last_known_addr { host: "127.13.160.189" port: 33749 } } peers { permanent_uuid: "22c9fb4ade074aa59c38d876d0ca1c19" member_type: VOTER last_known_addr { host: "127.13.160.188" port: 36405 } } }
I20260925 05:19:33.049276 4333 sys_catalog.cc:458] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0 [sys.catalog]: This master's current role is: LEADER
I20260925 05:19:33.050968 4344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c148d443950d40de90dd27c6827b6ae0: Generated new TSK 0
I20260925 05:19:33.051213 4344 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 05:19:33.085196 4153 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.083320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.13.160.190:41165)
0925 05:19:33.083372 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.083375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.084429 (+ 1054us) server_negotiation.cc:407] Connection header received
0925 05:19:33.084545 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.084546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.084558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.084591 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.085111 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33106: BlockingRecv error: recv got EOF from 127.0.0.1:33106 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 05:19:33.086057 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.083588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.13.160.189:33749)
0925 05:19:33.083631 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.083632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.085479 (+ 1847us) server_negotiation.cc:407] Connection header received
0925 05:19:33.085580 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.085582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.085595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.085627 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.085992 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40160: BlockingRecv error: recv got EOF from 127.0.0.1:40160 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:33.086826 4307 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.084141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.13.160.188:36405)
0925 05:19:33.084185 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.084187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.086312 (+ 2125us) server_negotiation.cc:407] Connection header received
0925 05:19:33.086411 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.086413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.086424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.086454 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.086763 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48820: BlockingRecv error: recv got EOF from 127.0.0.1:48820 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:33.092247 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.091277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.13.160.189:33749)
0925 05:19:33.091324 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.091326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.091571 (+ 245us) server_negotiation.cc:407] Connection header received
0925 05:19:33.091713 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.091714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.091738 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.091770 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.092180 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40166: BlockingRecv error: recv got EOF from 127.0.0.1:40166 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:33.098950 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.094787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.13.160.189:33749)
0925 05:19:33.094832 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.094834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.098187 (+ 3353us) server_negotiation.cc:407] Connection header received
0925 05:19:33.098315 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.098316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.098330 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.098362 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.098818 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40176: BlockingRecv error: recv got EOF from 127.0.0.1:40176 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 05:19:33.103513 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.102468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.13.160.189:33749)
0925 05:19:33.102527 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.102529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.102843 (+ 314us) server_negotiation.cc:407] Connection header received
0925 05:19:33.102988 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.102989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.103003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.103037 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.103446 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40184: BlockingRecv error: recv got EOF from 127.0.0.1:40184 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 05:19:33.109872 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.108899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.13.160.189:33749)
0925 05:19:33.108947 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.108950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.109240 (+ 290us) server_negotiation.cc:407] Connection header received
0925 05:19:33.109349 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.109350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.109363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.109394 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.109807 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40188: BlockingRecv error: recv got EOF from 127.0.0.1:40188 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 05:19:33.120388 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.119347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.13.160.189:33749)
0925 05:19:33.119394 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.119396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.119751 (+ 355us) server_negotiation.cc:407] Connection header received
0925 05:19:33.119863 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.119864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.119876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.119908 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.120311 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40204: BlockingRecv error: recv got EOF from 127.0.0.1:40204 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 05:19:33.139339 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.138109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40206 (local address 127.13.160.189:33749)
0925 05:19:33.138158 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.138160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.138521 (+ 361us) server_negotiation.cc:407] Connection header received
0925 05:19:33.138647 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.138649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.138662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.138696 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.139271 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40206: BlockingRecv error: recv got EOF from 127.0.0.1:40206 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 05:19:33.174026 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.172892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.13.160.189:33749)
0925 05:19:33.172945 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.172947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.173288 (+ 341us) server_negotiation.cc:407] Connection header received
0925 05:19:33.173425 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.173426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.173443 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.173481 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.173959 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40218: BlockingRecv error: recv got EOF from 127.0.0.1:40218 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 05:19:33.240324 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.239525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.13.160.189:33749)
0925 05:19:33.239592 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.239594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.239688 (+ 94us) server_negotiation.cc:407] Connection header received
0925 05:19:33.239748 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.239749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.239762 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.239833 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.240228 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40232: BlockingRecv error: recv got EOF from 127.0.0.1:40232 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 05:19:33.370389 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.369651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.13.160.189:33749)
0925 05:19:33.369721 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.369728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.369763 (+ 35us) server_negotiation.cc:407] Connection header received
0925 05:19:33.369843 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.369844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.369854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.369922 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.370326 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49206: BlockingRecv error: recv got EOF from 127.0.0.1:49206 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 05:19:33.628676 4230 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 05:19:33.627853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.13.160.189:33749)
0925 05:19:33.627927 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:33.627930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:33.627971 (+ 41us) server_negotiation.cc:407] Connection header received
0925 05:19:33.628084 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:33.628087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:33.628105 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:33.628181 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 05:19:33.628598 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49212: BlockingRecv error: recv got EOF from 127.0.0.1:49212 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 05:19:33.989972 4339 catalog_manager.cc:1259] Loaded cluster ID: de1c769ad7004538965ebe2fca8980bd
I20260925 05:19:33.990087 4339 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: loading cluster ID for follower catalog manager: success
I20260925 05:19:33.990936 4339 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: acquiring CA information for follower catalog manager: success
I20260925 05:19:33.991269 4339 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22c9fb4ade074aa59c38d876d0ca1c19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:19:34.048182 4363 catalog_manager.cc:1259] Loaded cluster ID: de1c769ad7004538965ebe2fca8980bd
I20260925 05:19:34.048285 4363 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: loading cluster ID for follower catalog manager: success
I20260925 05:19:34.049134 4363 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: acquiring CA information for follower catalog manager: success
I20260925 05:19:34.049467 4363 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f64d41d8880d471195663e39e29f2510: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 05:19:34.144523 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:34.141844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.13.160.189:33749)
0925 05:19:34.141977 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:34.141978 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:34.142037 (+ 59us) server_negotiation.cc:407] Connection header received
0925 05:19:34.142082 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:34.142083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:34.142133 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:34.142231 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:34.142369 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.142866 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.143780 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.143984 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.144041 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:34.144226 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:34.144337 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:34.144398 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:34.144432 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 05:19:34.144451 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260925 05:19:34.148880 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:34.146393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.13.160.188:36405)
0925 05:19:34.146521 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:34.146522 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:34.146531 (+ 9us) server_negotiation.cc:407] Connection header received
0925 05:19:34.146575 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:34.146575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:34.146613 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:34.146686 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:34.146781 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.147342 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.148157 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.148369 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.148422 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:34.148576 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:34.148691 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:34.148756 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:34.148787 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 05:19:34.148808 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260925 05:19:34.153573 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:34.150508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.13.160.189:33749)
0925 05:19:34.150559 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:34.150561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:34.150837 (+ 276us) server_negotiation.cc:407] Connection header received
0925 05:19:34.150940 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:34.150942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:34.150954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:34.151010 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:34.151178 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.151925 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.152833 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.153146 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.153194 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:34.153326 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:34.153354 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:34.153391 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:34.153444 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 05:19:34.153469 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:34.153869 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:34.150835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.13.160.188:36405)
0925 05:19:34.150874 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:34.150876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:34.151006 (+ 130us) server_negotiation.cc:407] Connection header received
0925 05:19:34.151127 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:34.151128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:34.151137 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:34.151183 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:34.151357 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.152075 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.153081 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.153378 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.153441 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:34.153571 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:34.153605 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:34.153659 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:34.153772 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 05:19:34.153799 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 05:19:34.154297 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 05:19:34.150651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.13.160.190:41165)
0925 05:19:34.150774 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 05:19:34.150777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 05:19:34.150784 (+ 7us) server_negotiation.cc:407] Connection header received
0925 05:19:34.150829 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 05:19:34.150830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 05:19:34.150908 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 05:19:34.150990 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 05:19:34.151086 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.151684 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.152725 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 05:19:34.152999 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 05:19:34.153092 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 05:19:34.153905 (+ 813us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 05:19:34.154053 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 05:19:34.154116 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 05:19:34.154164 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 05:19:34.154194 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260925 05:19:34.156198 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 4082
I20260925 05:19:34.161914 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 4157
I20260925 05:19:34.167196 13954 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskq3iYZX/build/release/bin/kudu with pid 4233
2026-09-25T05:19:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2848 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5702 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (273262 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST